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ABSTRACT 


AFIT/GSM/SM/77S-13 

a <x  ? 

The '-policy- capturing  model  frequently  used  in  psycho- 


logical research  was  adapted  in  this  study  "to  test  for  goal 
congruence  between  managers  at  different  levels  of  a large 
service  organization. of  the  U.S.  Air  Force.  J The  data  base 
was  derived  from  a field  experiment  wherein  660  military 
officers  and  civilian  managers  were  asked  to  make  decisions 
based  on  identical  information  cues.  The  decisions  were 
then  grouped  according  to  command  level  and  a multivariate 
linear  regression  model  was  derive^  for _each  ,group_:_^  An 

/•'"  u-ere  c & ' 

F-test  was  used  to  compare  ^he  se  models^for  equality  of  re- 
gression coefficients/  as  a determinant  of  the  effectiveness 
of  goal  communication  between  organizational  layers. — The 
linear  regression  model  proved'  readily  adaptable  as  an 
analytical  instrument  for  organizational  study.  The  results 
appeared  reasonably  conclusive,  with "significant  differences 

U/  f ^ 

in  decision-making  behavior,,  being  indicated  at  different 
command  levels. \ Finally,  a comparison  of  the  decisions  of 
senior  executives' against  those  of  various  middle  management 
groups  was  made  as  a test  of  the  importance  of  goal  con- 
gruence.  It  was  found  that  the /decisions  of  senior  managers 
and  subordinate  middle  management  groups,  made  under  identi- 
cal conditions,  differed  significantly  in  some  cases.  These 
differences  suggest  that  goal  congruence  within  the  Command 
might  have  considerable  importance,  and  that  it  might  be  im- 
proved through  a more  effective  management  information  sys- 
tem. 
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Chapter  I 


INTRODUCTION 

It  is  widely  held  among  organization  theorists  and 

management  specialists  that  organizations  are  goal  oriented. 

Several  excerpts  from  the  literature  on  organizations  serve 

to  illustrate  the  nearly  universal  appeal  of  this  assumption. 

Organizations  are  said  to  he: 

"...  [oriented]  toward  ends  that  are  objects  of  common 
knowledge."  (Simon,  1952,  p.  113°) 

"...  [aimed  at]  some  kind  of  collective  goal(s)  or 
output(s)."  (Leavitt,  1963,  p.  23) 

"...  devoted  primarily  to  attainment  of  specific 
goals."  (Etzioni,  1964,  p.  ix) 

"...  [structured  for]  obtaining  a set  of  objectives 
or  goals."  (Litterer,  1965,  p.  5) 

"...  [structured  toward]  achievement  of  some  common 
explicit  purpose  or  goal."  (Schein,  1970,  p.  9) 

The  assumption  of  goal  orientation  is  restated  so  often 
in  the  literature  that  this  assumption  is  accepted  as  axiom- 
atic by  most  students  of  organization  theory.  Indeed,  there 
seems  to  be  little  reason  to  doubt  that  implicit  goals  are 
inherent  to  any  organization;  but  most  authors,  as  shown 
above,  assume  that  organizations  are  oriented  toward  goals 
that  are  both  explicit  and  collectively  understood  by  the 
organization  membership.  Furthermore,  these  explicit  goals 
are  assumed  to  be  important  motivators  of  individual  behav- 
ior within  the  organization  (Porter,  Lawler  & Hackman,  1975. 
p.  78). 

The  assumption  of  explicit  goal  orientation  in  organiza- 
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tions  has  been  challenged  in  recent  years  by  several  authors. 
Monsen  and  Downs  (1965)  hypothesized  that  senior  managers  in 
large  organizations  are  motivated  not  by  organizational  goals 
but  by  the  pervasive  desire  to  maximize  their  individual 
lifetime  incomes  ("income"  being  measured  in  terms  of  money, 
power,  prestige,  or  other  personal  considerations) . This 
theory  is  extended  organizationally  downward  through  an  added 
precept  which  states  that  middle  managers  strive  to  please 
their  superiors,  rather  than  attempting  to  satisfy  organiza- 
tional goals.  Although  the  Monsen  and  Downs  theory  was  di- 
rected primarily  at  private-sector  organizations  thought  to 
be  profit-maximizing  entities,  there  is  evidence  suggesting 
that  the  theory  is  applicable  to  large  public-sector  organi- 
zations as  well  (Harrell,  1977).  Another  researcher  suggests 
that  the  process  of  identifying  and  communicating  organiza- 
tional goals,  which  should  be  most  exacting  and  carefully 
done,  is  more  often  than  not  a rather  haphazard  activity, 
because  organizations  are  not  inclined  to  devote  the  neces- 
sary resources  to  this  process  or  lack  the  objectivity  to  do 
the  job  well  (Manley,  1972,  p.  l). 

Regardless  of  an  author's  particular  persuasion  regard- 
ing goals  and  the  goal-setting  process,  it  seems  generally 
agreed  that  clear,  unambiguous  goals,  properly  communicated 
within  the  organization,  can  aid  substantially  in  promoting 
understanding  of  the  organizational  mission  and  in  develop- 
ing standards  upon  which  to  judge  the  performance  of  both 
people  and  units  of  activity. 
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The  Nature  of  Organizational  Goals 

Goals  have  been  generally  defined  by  organization 
theorists  as  desired  future  states  of  affairs  (Porter, 

Lawler  & Hackman,  1975.  p.  78).  While  goals  may  not  in  some 
cases  be  attainable,  organizations  are  viewed  in  general  as 
intending  to  reach  their  established  goals.  Herbert  Simon 
(1964,  p.  2)  classified  organizational  goals  into  two  cate- 
gories: official  goals  and  operative  goals.  Official  goals 
are  viewed  as  the  stated,  sometimes  cosmetic,  acceptable 
goals  of  the  organization,  and  they  may  be  either  very 
general  or  fairly  specific.  Some  authors  regard  official 
goals  as  being  primarily  for  external  consumption  (Manley, 
1972,  p.  3).  Operative  goals,  on  the  other  hand,  are  those 
goals  which  provide  the  bases  for  organizational  policy 
formulation,  operating  decisions,  development  and  applica- 
tion of  information  and  control  systems,  and  other  manage- 
ment functions.  Operative  goals  are  considered  to  be  in 
force  whether  or  not  there  exists  a conscious  organization- 
al goals-setting  process,  and  they  may  be  supportive  of, 
indifferent  to,  or  directly  opposed  to,  the  official  goals 
of  the  organization  (Manley,  1972,  p.  3) • 

Communicating  Organizational  Goals 

One  persistent  problem  in  large  organizations  is  the 
difficulty  of  accurately  communicating  the  goals  of  top 
management  to  those  who  must  implement  those  goals.  Such 
difficulty  can  stem  from  several  sources.  First,  the  com- 
munication process  itself  is  subject  to  erroneous  and  mis- 


3 


leading  input/output.  Second,  biases,  which  are  inherent  to 
all  groups  of  people,  can  limit  accuracy  in  reception  of 
managerial  intent.  Third,  the  views  of  groups,  when  con- 
tinually reinforced  by  group  members,  can  become  virtually 
immune  to  change  (Porter,  Lawler  & Hackman,  1975.  p.  378). 

An  example  of  the  latter  problem  might  be  the  continually 
reinforced  view  among  civilians  in  a military  organization 
that  management  goals  will  always  favor  military  personnel. 
Finally,  the  accuracy  of  communicating  top  management  goals 
can  be  adversely  influenced  by  the  "do  as  I say,  not  as  I 
do"  syndrome.  This  problem  might  develop  when  senior  man- 
agers designate  one  set  of  goals,  but  make  decisions  incon- 
sistent with  those  stated  goals. 

If  the  latter  situation  does  occur,  then  subordinates 
are  faced  with  the  difficulty  of  correctly  judging  the  op- 
erative goals  of  senior  managers  while  more  or  less  ignoring 
stated  goals.  In  his  Air  Training  Command  experiment, 

Harrell  (1975)  sought  to  test  the  ability  of  middle-level 
managers  to  correctly  interpret  the  desires  of  senior  man- 
agers when  those  seniors  acted  at  cross  purposes  to  the 
stated  goals  of  the  organization.  He  found  that  subordinate 
managers  readily  ignored  stated  goals  when  feedback  informa- 
tion revealed  that  their  superiors  had  done  so.  Furthermore, 
Harrell's  experiment  indicated  that  middle  managers  would 
change  their  decisions,  in  opposition  to  stated  policy, 
when  they  learned  that  their  superiors'  decisions  were  con- 
trary to  established  goals.  It  thus  became  obvious  that  an 


admonition  to  behave  in  a specified  fashion  is  insufficient 
to  guarantee  such  behavior  when  a contrary  example  is  set  by 
superiors. 

Since  Harrell's  experiment  was  conducted  entirely  with- 
in a single  organizational  level  of  the  Air  Training  Command, 
it  might  be  reasonable  to  conclude  that  it  would  be  even  more 
difficult  to  insure  an  accurate  perception  of  top  management 
goals  in  a widely  dispersed  organization.  Similarly,  the 
previously  cited  sources  of  difficulty  in  communicating 
organizational  goals  would  be  amplified  by  wide  geographical 
separation  of  units  within  an  organization. 

Goal  Congruence  Between  Organizational  Levels 

Goal  congruence  is  defined  by  Horngren,  a managerial 
accountant,  in  a systems  sense.  That  author  asserts  that  a 
managerial  system  must  provide  a global,  or  all-encompassing, 
emphasis,  so  that  major  goals  are  considered  whenever  man- 
agers act.  The  system  must  specify  goals  and  subgoals  to 
encourage  behavior  that  blends  with,  and  supports,  top  man- 
agement goals  (Horngren,  1972,  p.  155) • In  consonance  with 
this  systems  emphasis,  goal  congruence,  as  used  here,  is 
intended  tc  imply  general  agreement  between  the  decisions 
made  by  managers  at  all  organizational  levels,  given  the  same 
specified  set  of  goals  upon  which  to  base  those  decisions. 
Stated  another  way,  goal  congruence  may  be  said  to  exist  when 
different  managers  assign  the  same  relative  importance  to 
stated  goals  in  making  a specified  decision.  This  research 
effort  was  initiated  to  determine  whether  or  not  such  goal 
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congruence  exists  between  managers  at  different  organization- 
al levels  and  units  of  the  Air  Force  Communications  Service 
(AFCS),  a large  and  widely  dispersed  support  command  of  the 
United  States  Air  Force. 

AFCS  Command  Structure 

The  Air  Force  Communications  Service  is  organized 
essentially  as  shown  in  Figure  1-1.  Excepting  the  Northern 
and  Southern  Communications  Areas,  the  commanders  of  the  AFCS 
Areas  also  serve  as  Deputy  Chiefs  of  Staff  (or  Deputy  Com- 
manders) for  Communications-Electronics  for  the  major  com- 
mands they  support.  Due  to  differences  in  the  missions  sup- 
ported by  the  Areas,  each  Area  has  its  own  staff  structure, 
as  depicted  by  Figures  1-3  through  1-5*  Figure  1-2  shows 
the  basic  staff  structure  of  the  AFCS  command  headquarters. 
Inasmuch  as  AFCS  managers  theoretically  have  an  equal  chance 
of  being  assigned  to  any  of  the  Areas,  differences  in-  staff 
structure  are  not  considered  significant  for  purposes  of 
this  study  except  to  help  illustrate  the  diversity  of  the 
support  provided  by  AFCS.  The  wide  variety  of  combat  missions 
supported  by  the  Areas,  however,  is  significant,  as  is  the 
vast  geographical  dispersion  of  the  Command.  Figures  1-6 
through  1-8  serve  to  illustrate  both  the  geographical 
separation  of  the  operating  units  under  each  Area  and  the 
considerable  span  of  control  of  each  Area  commander.  The 
operating  units  vary  in  personnel  strength  from  less  than 
50  to  several  hundred.  The  total  strength  of  the  Command 
is  nearly  4-6,000,  making  it  one  of  the  six  largest  commands 
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in  the  Air  Force. 


Management  Information  Systems 

There  are  several  ways  in  which  AFCS  managers  receive 
and  dispatch  information  and  guidance.  In  addition  to  inter- 
faces between  headquarters  and  subordinate  unit  counterparts, 
conferences  are  held  at  the  Area  level  for  various  segments 
of  the  unit  management  staffs.  There  are  also  various  re- 
porting systems  which  assist  the  higher  command  echelons  in 
maintaining  visibility  and  control  over  field  unit  activi- 
ties. These  reporting  systems  tend  to  be  one-way  (upward) 
systems,  and  there  is  little  immediate  information  feedback 
to  the  reporting  units.  Furthermore,  there  is  an  absence  of 
lateral  information  transfer  systems;  hence,  the  operating 
groups  and  squadrons  do  not  frequently  exchange  information. 

The  Operating  Unit  Environment 

AFCS  operating  units  are  tenant  organizations  on  the 
bases  they  support.  In  effect,  AFCS  has  contracts  with  the 
various  mission  elements  of  the  Air  Force  to  provide  com- 
munications and  air  traffic  control  operating,  maintenance, 
planning,  programming,  engineering,  and  installation  ser- 
vices. These  agreements,  in  general,  require  that  AFCS  pro- 
vide trained  personnel  to  engineer,  install,  operate,  and 
maintain  communications-electronics  and  air  traffic  control 
facilities  which  are  the  real  property  of  the  using  command. 
The  methods  of  providing  these  services  are  in  many  ways  anal- 
ogous to  methods  employed  by  service  agencies  in  the  private 
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Rating  of  Operating  Units  and  Managers 

The  AFCS  field  operating  unit  (group,  squadron,  or 
detachment)  is  essentially  loaned  or  leased  to  the  using 
base  or  agency,  while  AFCS  retains  the  prerogative  of  man- 
aging the  personnel  resources  of  these  highly  technical  units. 
An  operating  unit  commander  typically  serves  two  masters: 
he  is  responsible  directly  to  his  next  higher  AFCS  commander, 
but  he  is  also  directly  responsible  to  the  base,  wing,  or 
other  commander  whom  his  unit  supports.  His  Officer  Effective- 
ness Report,  which  is  the  primary  instrument  by  which  he  is 
promoted  in  grade,  is  prepared  by  the  person  to  whom  his  unit 
provides  the  specified  services;  that  is,  the  non-AFCS  base 
commander  or  wing  commander  whom  the  AFCS  unit  has  been 
tasked  to  support.  The  Officer  Effectiveness  Report  is  then 
forwarded  to  the  appropriate  next  higher  AFCS  headquarters 
for  review  and  indorsement;  thus,  it  is  obviously  in  the 
best  interest  of  the  unit  commander  to  optimize  his  relation- 
ships with  both  his  local  superior  and  his  AFCS  superiors. 

It  seems  reasonable  to  hypothesize  that  pressures  external 
to  AFCS  might  be  expected  to  have  an  important  role  in  the 
response  of  the  field  unit  commanders  to  AFCS  goals  and  pol- 
icies . 

Figure  1-9  shows  the  internal  organization  of  a typical 
AFCS  field  unit  serving  a Strategic  Air  Command  (SAC)  base. 

This  unit  structure  changes  in  both  subtle  and  obvious  ways 
in  units  supporting  other  commands,  whose  mission  require- 
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ments  may  differ  widely  from  those  of  SAC.  These  differ- 
ences, along  with  the  geographical  separation  of  units  and 
varying  degrees  of  identification  of  AFCS  units  with  the 
supported  organizations,  tend  to  create  difficulty  in  com- 
paring AFCS  units  in  terms  of  performance.  Also  the  stan- 
dards against  which  the  effectiveness  and  efficiency  of  AFCS 
units  might  be  compared  tend  to  vary  as  the  policies,  pro- 
cedures, and  requirements  of  the  users  of  the  services  pro- 
vided. 

Objectives  of  the  Study 

This  study  was  conducted  under  the  sponsorship  of 
Headquarters,  Air  Force  Communications  Service,  Richards- 
Gebaur  Air  Force  Base,  Missouri.  It  was  intended  to  test 
for  goal  congruence  between  various  management  levels  of 
that  Command.  The  specific  objectives  of  the  study  were  as 
follows: 

a.  To  isolate  and  define  a set  of  goals  considered  by 
the  AFCS  headquarters  staff  to  be  important  to  effective  AFCS 
mission  accomplishment,  and  to  translate  these  goals  into 
specific  decision  criteria. 

b.  To  determine  the  relative  importance  placed  on  each 
of  the  above  goals  by  the  senior  AFCS  headquarters  staff  and 
by  other  managers  throughout  the  Command. 

c.  To  compare  the  values  thus  determined  and  to  ex- 
tract from  this  comparison,  if  possible,  some  measure  of 
the  degree  of  goal  congruence  between  the  various  managers 
and  managerial  groups. 
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Scope  and  Limitations  of  the  Study 

Because  a wide  spectrum  of  AFCS  management  personnel 
was  chosen  to  participate  in  the  data  collection  effort 
associated  with  this  study,  it  was  hoped  that  the  results 
of  the  research  might  be  indicative  of  the  decision  be- 
havior of  managers  throughout  the  Command.  It  is  not  ex- 
pected, however,  that  these  study  results  would  be  appli- 
cable to  other  Air  Force  commands  or  to  other  military  ser- 
vice organizations.  The  AFCS  mission  is  extremely  complex 
and  the  Command  employs  a highly  technical  work  force. 
Consequently,  management  of  the  Command  demands  skilled 
engineers,  computer  specialists,  mathematicians,  and  other 
well  educated  and  trained  managers.  Furthermore,  the  wide 
separation  of  AFCS  units  requires  managers  to  operate  with 
considerable  independence  and  provides  wide  latitude  for 
decision-making.  These  are  factors  not  frequently  found 
in  the  military  management  environment,  particularly  at  the 
middle  management  level. 
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Figure  1-1.  Organization  of  Air  Force  Communications  Service. 
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Figure  1-2.  A PCS  Headquarters  Staff  Organi za tion. 
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figure  1-5-  Tactical  Communl cations  Area  Staff  Organization. 
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Figure  1 -7.  Strategic  Air  Command  Communications  Area  Unit  Composition 


Chapter  II 

MODELING  HUMAN  JUDGEMENT 

It  has  been  argued  frequently  and  vehemently  that  hu- 
man judgement  is  too  complicated  to  predict  with  any  real 
accuracy,  and  this  might  well  prove  true  if  one  attempted 
to  build  a model  for  predicting  the  response  of  a partic- 
ilar  individual  to  non-specific  external  stimuli.  Argu- 
ments citing  the  complexity  of  the  human  decision  process 
appear  to  stem  from  several  factors.  First,  the  utiliza- 
tion of  external  stimuli,  or  cues,  to  arrive  at  decisions 
seems  to  be  dependent  upon  numerous  environmental,  educa- 
tional, and  experiential  behavior  elements  learned  over 
time  by  every  individual.  Second,  the  process  of  decision- 
making, or  judgement,  is  apparently  not  a purely  intellec- 
tual one,  or  even  an  intellectual/emotional  process  alone. 
Judgement  also  seems  to  involve  physiological  dynamics, 
such  as  the  individual's  state  of  physical  health,  his 
diet,  and  a plethora  of  chemical/hormonal  balances  and 
imbalances.  Finally,  the  human  judgement  process  appears 
to  incorporate  varying  degrees  of  intuition,  ranging  from 
nearly  logical  inferences  to  insights  that  are  hardly  short 
of  incredible. 

Despite  the  arguments  against  attempting  to  simulate 
or  model  the  human  judgement  process,  there  are  numerous 
authors  who  believe  that  this  problem  does  not  involve 
modeling  the  complex  decision  process  itself,  but  only  the 
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manner  in  which  humans  use  available  information  in  making 
decisions.  Consequently,  the  attention  of  many  specialists 
(socia-L  scientists,  in  particular)  has  begun  to  focus  on  the 
process  of  integrating  information  for  the  purpose  of  pre- 
dicting human  judgement.  In  the  opinion  of  some  experts  the 
real  issues  in  simulating  human  judgement  are  the  questions 
of  what  the  individual  does  with  available  information  and 
what  he  should  be  doing  with  that  information.  The  first 
question  involves  the  psychology  of  how  individuals  use  in- 
formation while  the  second  question  seems  to  be  more  practi- 
cal, involving  the  attempt  to  make  the  decision  process  more 
efficient  (Slovic  & Lichtenstein,  1971,  p.  652). 

A number  of  mathematical  models  have  been  proposed  for 
capturing  human  judgement  policies.  The  model  used  in  this 
study  was  the  linear  model  suggested  by  Hoffman  (i960), 
which  is  based,  in  turn,  upon  the  lens  model  presented  by 
Brunswick  (1952).  Without  reiterating  the  details  of  the 
work  of  Hoffman  and  Brunswick,  whose  efforts  are  well-known 
in  the  field  of  psychology,  let  it  suffice  to  say  that  the 
linear  model  is  founded  upon  the  notion  that  a judge's  de- 
cisions represent  linear  combinations  of  the  available  stim- 
uli, or  cues. 

Linear  Regression  Models 

The  work  of  Brunswick,  Hoffman,  and  many  others  inter- 
ested in  modeling  human  judgement  is  based  upon  the  process 
of  fitting  data  to  a linear  equation  through  the  method  of 
least  squares  regression.  The  resulting  equation  is  referred 
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to  as  a regression  model.  It  contains  a criterion  vari- 
able and  one  or  more  predictor  variables  in  a linear  com- 
bination described  mathematically  by  coefficients  which 
are  derived  through  the  least  squares  procedure.  Given 
specified  information  cues,  the  criterion  can  be  predicted 
from  linear  combinations  of  these  cues  by  the  following 
general  regression  equation: 

1 Y = 30  + + S32X2  + Mi  + e 

where  the  coefficients  (S0,  3^,  i = 1,  2,  . . . , k)  are  the 
parameters  of  the  model  and  6 is  an  error  term  which  ac- 
counts  for  any  variance  in  Y not  explained  by  the  predic- 
tors (X^_)  in  the  least  squares  regression.  It  should  be 
noted  that  the  term  "linear"  refers  to  linearity  in  the 
parameters  of  the  regression  model  and  not  to  the  predic- 
tors; hence,  a regression  equation  such  as 

y = 30  + Ml  + Ml  + € 

is  a second-order  (in  X)  linear  (in  the  3's)  regression 
model  (Draper  & Smith,  1966,  p.  9). 

In  practice,  it  is  not  usually  feasible  to  examine  all 
possible  occurrences  of  Y and  X;  therefore,  statistical 
sampling  is  used  to  derive  estimates  of  the  actual  para- 
meters of  the  regression  model.  When  judgement  is  repre- 
sented by  the  linear  regression  model  the  estimates  of  the 
parameters  thus  derived,  usually  denoted  as  bQ , b^ , ...,  b^, 
are  used  to  represent  the  relative  importance  given  to  each 
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cue  by  the  judge  in  arriving  at  his  decision  (Y)  (Slovic  & 
Lichtenstein,  1971,  p.  65 9).  Hoffman  (I960)  proposed  an 
alternative  index,  which  he  called  "relative  weight",  de- 
signed to  better  represent  the  importance  placed  on  each 
cue  by  the  judge.  Relative  weights  are  calculated  from  the 
parameters  of  the  regression  model  and  the  squared  multiple 
correlation  coefficient  (R2)  derived  from  the  regression, 
as  follows: 


RW. 


iY 


blYriY 

Rv 


where  RWyy  is  the  relative  weight  of  the  predictor  Xy  with 
respect  to  the  decision  Y (the  regression  model  criterion 
variable),  byy  is  the  standardized  regression  coefficient 
(usually  called  the  beta-weight)  yielded  by  the  least 
squares  regression  of  Y with  the  predictor  X^,  and  r^y  is 
the  intercorrelation  coefficient  between  the  predictor  vari- 
able Xy  and  the  decision  Y.  Ry  is,  as  indicated  above,  the 
squared  multiple  correlation  coefficient  for  the  regression. 
This  latter  coefficient  indicates  the  proportion  of  variance 
in  the  decision  Y which  is  explained  by  the  regression. 

As  will  be  seen  later,  the  policy-capturing  technique 
employed  for  this  study  involves  the  use  of  orthogonal 
(independent)  predictors  in  the  regression  equation.  In 
this  special  case,  the  intercorrelation  coefficients  and 
the  beta-weights  resulting  from  the  regression  are  identical. 
The  formula  for  calculating  the  relative  weights  of  the  pre- 
dictors, therefore,  reduces  to: 
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Examples  of  Linear  Modeling  Applications 

There  is  considerable  justification  in  the  literature 
for  use  of  the  linear  model  as  a predictor  of  decisions  by 
human  judges.  Slovic  and  Lichtenstein  (1971.  p.  679)  cite 
more  than  30  studies  where  researchers  have  very  successful- 
ly represented  various  judges'  idiosyncratic  weighting  pol- 
icies with  linear  models.  Such  studies  are  tremendously 
diverse  in  the  judgemental  tasks  to  which  the  linear  model 
has  been  applied.  The  tasks  include  judgements  about  per- 
sonality characteristics,  performance  in  school  or  on  the 
job,  physical  and  mental  pathology,  legal  decisions,  and 
various  gambles,  such  as  attractiveness  of  common  stocks. 

In  some  cases  the  cues  supplied  were  artificial  and  the 
judges  were  unfamiliar  with  the  task.  An  example  of  this 
is  a study  where  college  students  were  asked  to  judge  the 
intelligence  of  their  peers  on  the  basis  of  grade  point 
average,  aptitude  test  scores,  and  other  such  cues  (Knox  & 
Hoffman,  1962).  In  other  cases,  studies  were  made  of  judge- 
ments in  complex  but  familiar  situations  by  skilled  decision- 
makers who,  in  addition  to  the  cues  included  in  the  predic- 
tion equation,  had  other  information  available  to  them.  The 
following  excerpt  provides  examples  of  such  cases: 

"...  Kort  (1968)  modeled  judicial  decisions  in  work- 
men's compensation  cases  using  various  facts  from 
the  cases  as  cues.  Brown  (1970)  modeled  casework- 
ers' suicide  probability  estimates  for  persons  phon- 
ing a metropolitan  suicide  prevention  center;  the 


24 


cues  were  variables  such  as  sex,  age,  suicide  plan, 
etc.,  obtained  from  the  telephone  interview.  And 
Dawes  (1970)  used  a linear  model  to  predict  the 
ratings  given  applicants  for  graduate  school  by 
members  of  the  admissions  committee."  (Slovic  & 
Lichtenstein,  1971,  p.  679) 


Further  Rationale  for  Use  of  Linear  Decision  Models 

Information  processing  by  humans  in  the  face  of  un- 
certainty has  long  been  a concern  of  psychologists  and  other 
social  scientists.  Many  studies  have  been  conducted  which 
used  various  aspects  of  statistics  as  normative  models  in 
the  investigation  of  human  behavior.  The  philosophical 
basis  for  such  studies  is  that  man  must  often  resort  to 
use  of  equivocal  or  probabilistic  information  in  making  de- 
cisions about  real  world  situations  (Beach,  1967,  p.  276). 

In  other  words,  man  must  frequently  make  judgements  re- 
garding situations  about  which  he  has  incomplete  informa- 
tion. 

In  decisions  involving  uncertainty,  man  may  not  always 
be  as  good  a judge  as  the  model  of  man.  For  example,  one 
study  involved  capturing  the  diagnostic  policies  of  29 
clinical  psychologists  and  forming  a composite  linear 
model  of  those  policies.  It  was  found  that  the  composite 
model  was  nearly  always  more  accurate  than  the  individual 
human  judges  in  drawing  clinical  inferences  because  of  the 
elimination  of  a significant  amount  of  the  random  error  to 
which  the  individual  clinicians  were  prone  (Goldberg,  1970, 
p.  476).  In  another  study,  linear  models  were  constructed 
of  80  university  judges'  policies  for  predicting  graduate 
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student  grade  point  averages  based  on  undergraduate  grades, 
Graduate  Record  Examination  scores,  and  other  cues.  These 
linear  models  were  then  used  to  forecast  actual  graduate 
grade  point  averages  and  the  results  were  compared  against 
first-hand  predictions  by  the  judges.  In  every  case,  the 
model  proved  superior  to  the  judge  from  whom  it  was  derived 
(Wiggins  & Kohen,  1971.  P*  105). 

The  results  of  the  above  studies  and  numerous  others 
show  clearly  that  the  linear  model  is  a powerful  device  for 
predicting  quantitative  judgements  arrived  at  on  the  basis 
of  specific  cues;  so  much  so,  in  fact,  that  many  researchers 
seem  to  have  concluded  that  the  model  is  better  at  making 
decisions  than  the  man  being  modeled.  It  seems  reasonable 
to  conclude,  on  the  basis  of  this  evidence,  that  the  linear 
regression  model  might  also  be  used  as  an  effective  analyti- 
cal instrument  for  purposes  such  as  the  type  of  study  describ- 
ed here. 


Chapter  III 


RESEARCH  METHODOLOGY 

Identification  of  AFCS  Goals 

Since  the  objective  of  this  study  was  to  test  for 
goal  congruence  it  was  necessary  at  the  outset  to  identify 
a set  of  organizational  goals  which  met  several  criteria, 
as  follows:  (l)  the  goals  must  be  realistic,  operative  goals 
of  the  Command,  (2)  the  goals  must  be  unambiguous  and  reason- 
ably independent  of  one  another,  and  (3)  the  accomplishment 
of  the  goals  must  be  measurable. 

Four  AFCS  goals  which  met  the  above  requirements  were 
identified.  First,  AFCS  strives  to  provide  personnel  support 
programs  to  maximize  retention  of  its  skilled  technical 
personnel,  to  maintain  high  morale  and  esprit  de  corps,  to 
minimize  disciplinary  and  complaint  rates,  and  to  provide 
adequately  for  the  welfare  of  the  AFCS  work  force.  Second, 
the  Command  seeks  to  maximize  the  quality  of  the  service  it 
provides  to  its  customers.  Third,  compliance  with  AFCS  di- 
rectives is  sought  in  this  Command  whose  field  units  are 
widely  separated  and  often  far  removed  from  supervising 
staffs.  Finally,  the  Command  seeks  maximum  effectiveness 
and  efficiency  in  repair  and  preventative  maintenance  of  the 
facilities  for  which  it  has  responsibility. 

Design  of  the  Experimental  Task 

It  was  proposed  in  this  study  to  employ  a policy- 
capturing technique  similar  to  that  suggested  by  Christal 
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(1968b)  and  used  by  Harrell  (1975)  in  his  Air  Training  Com- 
mand experiment.  Accordingly,  the  above  listed  goals  were 
translated  into  four  predictor  criteria  for  an  experimental 
judgement  task  from  which  a linear  model  could  be  derived 
for  each  managerial  entity  to  be  tested. 

Recalling  the  form  of  the  basic  linear  regression 
model,  and  dealing  henceforth  only  with  the  model  applicable 
to  the  AFCS  sample  population,  it  would  be  well  to  address 
at  this  point  the  effect  of  independence  in  predictor  cri- 
teria. The  appropriate  linear  model  for  this  study  is 

Y = b1X1  + b2X2  + b ytj  + b^X^  + € 

where  X^  through  X^  are  the  predictor  variables  translated 
from  the  selected  AFCS  goals,  as  follows: 

X^  = Personnel  Programs 
X2  = Quality  of  Service 
X^  = Compliance 
X^  = Maintenance  Quality 

The  intention,  following  Harrell's  1975  example,  was 
to  devise  an  experimental  task  where  AFCS  judges  are  asked 
to  evaluate,  on  an  eight  point  scale,  the  performance  of 
hypothetical  operating  squadrons.  Each  decision  case  con- 
tains information  about  whether  each  predictor  variable  is 
rated  satisfactory  or  unsatisfactory.  Since  there  are  four 
predictors  (which  may  be  referred  to  as  unit  performance 
criteria)  and  only  two  states  for  each  criterion  (satis- 
factory or  unsatisfactory),  there  are  2^  = 16  possible 
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combinations  to  consider;  hence,  each  selected  AFCS  judge 
was  asked  to  rate  the  performance  of  16  hypothetical  squad- 
rons. 

The  use  of  satisfactory/unsatisfactory  values  for  the 

judgement  cues  consigns  the  researcher's  model  to  special 

■**» 

status,  since  the  sixteen  decision  tasks  can  each  be  rep- 
resented by  a mathematically  orthogonal  vector.  This 
assures  that  the  predictors  themselves  will  be  uncorrelated 
in  the  regression,-  which  in  turn  provides  that  the  "use- 
fulness" of  the  performance  criteria  in  arriving  at  the 
performance  evaluation  is  strictly  measured  by  the  inter- 
correlation between  each  performance  criterion  and  the  de- 
cision (Darlington,  1968,  p.  162).  In  other  words,  each 
of  the  sixteen  evaluations  made  by  the  experimental  subject 
is  strictly  characterized  by  the  numerical  value  (1,  2,  .... 
8)  of  his  evaluation  and  one  of  sixteen  orthogonal  pre- 
dictor vectors,  each  of  which  represents  a unique  perfor- 
mance state.  It  is  thus  quite  simple  to  determine  the  im- 
portance the  subject  places  on  each  of  the  four  performance 
criteria. 

Figure  3-1  clearly  illustrates  the  orthogonality  of 
the  sixteen  decision  cases  represented  by  assignment  of 
ratings  of  S or  U to  a set  of  four  decision  criteria.  Each 
of  the  digital  equivalents  of  the  sixteen  cases  can  be  read 
directly  into  the  regression  algorithm  as  a unique  predic- 
tor vector. 

The  data  collection  instrument  used  for  this  study  is 
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Case 

Number 

Criterion 
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Predictor 

Vectors 

1 
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Figure  3-1*  Orthogonal  Predictor  Vectors 
Associated  With  Ratings  of  S 
or  U for  a Set  of  Four  Deci- 
sion Criteria. 
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presented  in  Appendix  B.  The  sixteen  cases  were  randomly 
arranged  in  the  exercise  booklet  by  assigning  an  arbitrarily 
selected  random  number  from  a random  number  table  to  each 
case,  with  the  cases  listed  as  shown  in  Figure  3-1,  then 
sequentially  listing  the  random  numbers  and  the  cases  asso- 
ciated with  each  number. 

Collection  of  the  Experimental  Data 

In  agreement  with  AFCS , the  researcher  chose  middle 
management  personnel  at  the  Command  headquarters , Northern 
Communications  Area  (NCA) , Strategic  Air  Command  Communica- 
tions Area  (SACCA) , Tactical  Communications  Area  (TCA) , and 
the  operating  units  under  each  Area  for  participation  in 
this  experiment.  Middle  managers  were  defined  as  Air  Force 
officers  in  the  grades  of  lieutenant  through  lieutenant 
colonel  and  civilian  managers  in  the  grades  GS-9  through 
GS-13.  It  is  this  management  segment  which  is  largest  in 
number  and  which  has  the  greatest  responsibility  for  insur- 
ing that  AFCS  organizational  goals  are  met. 

A sample  of  80  managers  was  selected  at  each  head- 
quarters except  TCA,  where  the  entire  population  of  63 
middle  managers  was  asked  to  participate.  The  experimental 
exercises  were  delivered  personally  by  the  author  to  each 
of  the  Areas  and  to  the  Command  headquarters.  A project 
officer  was  designated  at  each  of  these  agencies  to  col- 
lect the  completed  instruments  and  return  them  by  mail  to 
the  author. 

A total  of  360  operating  unit  personnel  was  selected 


1 


to  participate  in  the  exercise.  These  subjects  received  the 
experimental  exercise  by  mail  and  were  asked  to  return  the 
completed  instruments  in  self-addressed  envelops  provided 
for  that  purpose.  A lower  response  rate  was  anticipated 
for  the  operating  units  because  of  the  requirement  to  mail 
the  data  collection  instruments;  hence,  a slightly  higher 
proportion  of  potential  subjects  was  selected  from  each  of 
these  populations  than  from  those  of  the  headquarters 
staffs . 

The  randomness  of  subject  selection  from  each  popula- 
tion was  assured  by  the  following  process: 

a.  The  AFCS  Deputy  Chief  of  Staff/Personnel  prepared 
computer  listings  of  middle  managers  assigned  to  the  Command 
headquarters  and  each  of  the  three  Areas.  Similarly,  lists 
were  generated  which  contained  the  names  of  management 
personnel  assigned  to  the  operating  units,  grouped  by  Area. 
Thus,  a total  of  seven  separate  listings  was  created. 

b.  For  each  of  the  seven  listings  provided  a random 
number  was  chosen  from  a random  number  table  by  arbitrary 
prior  selection  of  a line  and  column  number  (e.g.,  line 
column  12).  This  yielded  a 5-digit  random  number  from  which, 
also  by  prior  arbitrary  choice,  a two-digit  number  was  se- 
lected (e.g.,  first  two  digits,  last  two  digits,  middle  two 
digits).  The  two-digit  number  thus  selected  was  used  as 

the  starting  point  on  the  personnel  listing  and  from  that 
point  every  nth  person  was  selected,  n being  equal  to  the 
total  number  on  the  list  divided  by  the  desired  sample  size. 
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Selections  were  continued  by  counting  on  through  the  top 
of  the  list  as  necessary. 

Coding  of  Collected  Data 

When  the  completed  data  collection  instruments  were 
returned,  the  information  from  each  one  was  coded  onto  a 
standard  IBM  card.  Figure  3-2  shows  the  card  format  and 
variable  names  used  for  the  data  base.  The  demographic  data 
for  each  judge  were  collected  for  use  in  a separate  but  re- 
lated research  effort  being  conducted  simultaneously  with 
this  study.  The  name  of  the  subject  was  recorded  in  the 
data  base  only  if  he  requested  an  analysis  of  his  performance 
in  the  exercise,  which  was  offered  as  an  incentive  to  in- 
crease the  return  rate.  An  example  of  the  feedback  provided 
to  those  requesting  it  is  presented  in  Appendix  C. 

Restructuring  the  Data  for  Regression  Analysis 

Each  coded  data  card  contained  the  numerical  values 
(1,  2 8)  for  every  ’unit  evaluation  made  by  the  sub- 

ject, but  did  not  contain  the  predictor  vector  associated 
with  each  of  the  16  decisions.  Also,  the  16  evaluations 
were  coded  on  the  cards  in  a horizontal  format,  and  the 
CDC  6600  computer  regression  algorithm  available  to  the 
researcher  requires  that  the  regression  variables  be  read 
in  line  by  line.  Therefore,  it  was  necessary  to  restructure 
the  data  for  regression  analysis.  To  accomplish  this,  a 
FORTRAN  program  was  prepared  which  reads  the  numerical 
values  for  the  16  decisions  coded  on  each  card  and  creates 
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an  n x 5 matrix  for  storage  of  each  decision  and  its  re- 
spective predictor  vector,  n being  16  times  the  number  of 
subjects  responding.  The  program  adds  the  appropriate  pre- 
dictor vector  to  each  evaluation,  then  stores  the  entire 
matrix  thus  created,  along  with  all  the  demographic  data 
for  each  subject,  in  a computer  disk  file.  This  disk 
file  is  then  attached  by  the  regression  program  as  required. 
The  FORTRAN  program  was  prepared  by  Major  C.W.  McNichols,  a 
professor  in  the  AFIT  Systems  Management  Department,  and 
has  general  application  possibilities  for  other  studies  of 
this  nature. 

Additional  Data  Collected 

In  addition  to  the  66 3 data  collection  instruments 
distributed  to  AFCS  middle  managers,  the  senior  managers 
of  the  Command  were  also  requested  to  provide  an  input  to 
this  study.  The  AFCS  Chief  of  Staff  and  each  of  seven 
deputy  chiefs  of  staff  was  requested  to  provide  the  re- 
searcher with  the  weight  he  felt  should  be  given  to  each 
of  the  four  performance  criteria  in  evaluating  AFCS  units. 
This  was  accomplished  by  asking  each  senior  manager  to 
divide  100  points  between  the  four  criteria.  Then,  each 
of  these  senior  executives  was  asked  to  complete  the  per- 
formance evaluation  task  in  the  same  manner  as  all  other 
subjects.  The  objective  of  this  request  was  to  provide  a 
means  for  comparing  the  perceived  and  actual  importance 
assigned  to  the  performance  criteria  by  the  most  senior 
managers  of  the  Command  with  like  data  calculated  from  the 
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responses  of  the  middle  managers.  An  example  of  the  ex- 
planatory letter  sent  to  each  senior  manager  is  contained 
in  Appendix  B.  Precautions  were  taken  to  protect  the 
privacy  of  the  senior  respondents  to  this  request,  as  shown 
in  the  sample  explanatory  letter. 

Analysis  of  the  Experimental  Data 

The  analysis  medium  used  for  this  study  was  the  Aero- 
nautical Systems  Division  (ASD)  CDC  6600  computer,  employ- 
ing the  Statistical  Package  for  the  Social  Sciences  (SPSS) 
multivariate  regressior  Lgorithm  (Nie,  et  al.  , 1975)*  Al- 
so, a number  of  FORTRAN  programs  were  written  by  the  author 
to  facilitate  use  of  the  data  output  from  the  SPSS  regres- 
sion runs.  A program  was  written  to  calculate  the  relaxive 
weights  of  the  predictors  in  every  regression  and  to  compute 
the  F-test  value  for  comparing  the  regression  models  in  each 
composite  run.  Another  program  was  written  to  select  data 
from  the  disk  file  and  prepare  the  individual  analyses  for 
subjects  requesting  such  analysis.  Finally,  the  author  modi- 
fied another  program,  written  by  Major  C.  W.  McNichols,  to 
calculate  the  relative  weights  and  multiple  correlation  co- 
efficient for  every  respondent  to  the  experimental  exercise. 

The  SPSS  regression  algorithm  is  extremely  flexible, 
providing  several  types  of  regression,  tests  for  signifi- 
cance, and  various  statistics.  The  data  for  this  study 
were  subjected  to  step-wise  regression  in  every  case,  and 
in  every  regression  the  evaluation  variable  (the  subject's 
evaluation  of  unit  performance)  was  regressed  on  through 
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(the  unit  performance  criteria,  listed  in  the  order 
presented  earlier) . Appendix  D contains  examples  of  the 
SPSS  regression  program  used  and  typical  outputs  from  the 
regression  runs  completed  for  this  study.  A total  of  28 
separate  regression  models  was  generated  in  analyzing  the 
data  for  this  study,  hut  this  was  easily  accomplished  with 
the  useful  selection  features  of  the  SPSS  algorithm. 

To  assure  generation  of  all  the  models  required  to 
address  the  basic  research  question,  the  matrix  shown  in 
Figure  3-3  was  developed.  The  numbers  along  the  horizontal 
and  vertical  sides  of  the  matrix  represent  the  9 different 
regression  groups  listed  at  the  bottom  of  the  figure.  A 
regression  model  was  generated  for  every  ncn-zero  element 
in  the  matrix.  Where  a non- zero  element  appears  at  the 
intersection  of  like-numbered  rows  and  columns,  the  associ- 
ated regression  run  included  only  members  of  that  one  group. 

All  other  non-zero  elements  indicate  composite  groups  in- 
cluded in  the  regression  run.  It  is  necessary  to  regress 
two  groups  together  only  once;  therefore,  there  are  no 
reversals  indicated  in  the  matrix  (i.e.,  all  elements  below 
the  diagonal  are  zero). 

Testing  for  Statistical  Differences  Between  Models 

Each  of  the  28  regressions  indicated  above  produced 
a model  which  was  used  to  determine  whether  a statistically 
significant  difference  exists  between  two  or  more  groups  of 
AFCS  managers.  A statistically  significant  difference  would 
indicate  that  the  regression  coefficients  represent  different 
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populations  and,  therefore,  that  the  importance  given  to 
the  performance  criteria  by  at  least  one  group  in  the  com- 
parison is  significantly  different  from  the  others.  The 
F-test  used  for  comparisons  in  this  study  is  described  by 
G.  C.  Chow  (i960,  p.  599) • The  null  hypothesis  tested  in 
each  comparison  is  that  there  is  no  significant  difference 
in  the  regression  coefficients  of  the  models  being  compared. 

The  alternate  hypothesis  is  that  there  is  at  least  one 
model  among  those  being  compared  whose  regression  coeffi- 
cients are  significantly  different  from  the  others.  In  other 
words,  statistically  different  populations  are  involved.  Re- 
jection of  the  null  hypothesis,  for  purposes  of  this  study, 
is  tantamount  to  stating  that  there  are  significant  dif- 
ferences in  the  goals  of  the  groups  being  compared.  The 
level  of  significance  used  for  all  F-tests  applied  for  this 
study  was  0.05*  The  method  of  calculating  the  observation 
F value  for  testing  the  null  hypothesis  is  shown  in  Appendix 
E. 

2 . 

Significance  of  R in  Orthogonal  Predictor  Models 

One  further  aspect  of  models  having  orthogonal  pre- 
dictors is  important  to  this  study,  and  that  is  the  signi- 
2 

ficance  of  R , the  squared  multiple  correlation  coefficient 
for  the  regression.  If  a regression  equation  is  generated 
for  each  individual  judge  making  the  same  decision,  using 
the  same  information  cues,  such  an  equation  expresses  com- 
pletely the  policy  of  that  judge  (Christal,  1968a,  p.  26). 

. 2 

Computation  of  the  R for  each  such  model  is  then  an 
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effective  measure  of  the  prediction  efficiency  of  the  judge, 
since  variations  about  the  predicted  value  of  the  decision 
variable  are  largely  due  to  inconsistencies  in  application 
of  the  judge's  policy. 

When  the  predictors  are  uncorrelated,  as  in  the  case 

2 

of  the  models  used  for  this  study,  the  value  of  R can 
actually  be  viewed  as  a measure  of  the  judge's  consistency 
in  applying  his  established  weights  to  the  predictors  in 
making  his  l6  evaluations.  This  notion  can  be  extended  to 
models  representing  larger  groups  also,  but  it  becomes  rela- 
tively more  obscure  as  judges  are  clustered  together,  for  the 
following  two  reasons:  (l)  a greater  number  of  random  errors 
is  introduced  in  larger  groups  of  judges,  and  (2)  each  judge 
may  be  very  consistent  in  applying  his  own  policy,  but  the 
composite  group  may  not  reflect  the  same  degree  of  consis- 
tency as  the  individual  judges  (will  not,  in  fact). 

• 2 

The  notion  of  using  R as  a measure  of  rater  consis- 
tency was  important  for  evaluating  the  ARCS  senior  manager 
contributions  to  this  study,  and  to  providing  feedback  to 
those  experimental  subjects  who  requested  it. 
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Chapter  IV 


RESULTS  OF  THE  EXPERIMENT 

AFCS  Response  to  the  Experiment 

As  shown  in  Table  4-1,  of  a total  of  6?1  experimental 
exercises  distributed  to  AFCS  managers  520  usable  completed 
exercises  were  returned  for  analysis,  resulting  in  an  over- 
all response  rate  of  77 • 5$*  As  anticipated,  the  response 
rates  among  the  operating  units  was  somewhat  lower  than  in 
the  several  headquarters  agencies,  where  managers  were  con- 
centrated in  larger  numbers  and  could  be  reached  directly 
rather  than  requiring  mail  distribution  of  the  exercise. 

Managers  contacted  directly  during  this  experiment  al- 
most invariably  expressed  keen  interest  in  the  research. 
There  were  also  numerous  expressions  of  interest  in  the  form 
of  written  comments  and  suggestions  which  were  returned  with 
the  exercises.  The  high  interest  of  the  Command  is  most 
notably  reflected,  however,  in  the  349  requests  for  analy- 
ses of  individual  performance  in  the  exercise,  a total 
which  constitutes  68$  of  the  exercise  respondents. 

Senior  Manager  Results 

The  seven  returns  from  AFCS  senior  managers  were  sub- 
jected to  separate  step-wise  regressions,  then  grouped  to- 
gether in  a composite  regression  run.  The  results  are  dis- 
played in  Table  4-2. 

The  senior  managers  were  extremely  consistent  in 
applying  their  individual  evaluation  policies,  as  evidenced 
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Table  4-1.  Experimental  Exercise  Distribution  and  Response 


Management 

Group 

Rates . 

Number  of 
Exercises 
Distributed 

Number 
of  Usable 
Returns 

Response  Rate 

Senior  Staff 

8 

7 

87.5% 

Hq  AFCS 

80 

70 

87-5% 

Hq  NGA 

80 

62 

71.5% 

Hq  3ACCA 

80 

69 

86.25% 

Hq  TCA 

63 

58 

92.0 6% 

NCA  Units 

134 

102 

76.12% 

SACCA  Units 

119 

35 

71.43% 

TCA  Units 

107 

67 

62 . 62% 

Total 

671 

520 

77-5% 

^ 

p 

by  the  mean  R of  0. 91366;  however,  it  is  apparent  that 
there  are  some  significant  differences  in  the  policies  of 
these  senior  executives.  One  senior  manager  (Number  l) 
placed  k2%  of  his  evaluation  emphasis  on  the  Compliance  fac- 
tor. Another  (Number  5)  weighted  the  Quality  of  Service 
factor  at  only  21%  • These  two  findings  are  unusual,  for 
AFCS  managers  who  participated  in  the  experiment  general- 
ly rated  the  Quality  of  Service  factor  far  higher  than  any 
other,  while  the  Compliance  factor  was  generally  rated  quite 
low  (10-l^S) 

Another  interesting  aspect  of  the  senior  manager  anal- 
ysis is  the  comparison  of  the  weights  each  manager  indicated 
he  believed  should  be  applied  to  the  predictors  versus  the 
weights  he  actually  used  in  his  unit  evaluations.  Of  the 
five  senior  managers  who  submitted  their  perceived  weights 
for  the  predictors,  only  one  (Number  2)  actually  cane  close 
to  applying  those  indicated  weights.  In  most  cases,  the 
Quality  of  Service  motive  received  significantly  higher  con- 
sideration in  the  exercise  than  the  executives  indicated 
they  felt  this  factor  should  receive.  This,  of  course,  re- 
sulted in  compensating  differences  in  the  weights  given  to 
the  remaining  predictors.  This  tendency  to  apply  different 
weights  than  those  specified  seems  to  indicate  that  a valid 
comparison  has  been  made  between  the  stated  and  operative 
goals  of  senior  AFCS  executives,  serving  to  demonstrate  how 
the  "do  as  I say,  not  as  I do"  syndrome  can  arise.  There 
appears  to  be  little  question  that  the  goals  used  by  senior 
AFCS  executives  to  formulate  Command  policy  can  represent 
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Table  4-2.  Senior  Manager  Analysis  Results, 


Senior 

Manager 

Predictor 

Beta 

Weight 

1 

Personnel 

Service 

Compliance 

Maintenance 

.3077935 

.5129892 

.6155870 

.4103913 

2 

Personnel 

Service 

Compliance 

Maintenance 

•5013947 

.5682473 

.3008368 

•5632473 

3 

Personnel 

Service 

Compliance 

Maintenance 

.2961898 
.72908 26 
•3189736 
.5012443 

4 

Personnel 

Service 

Compliance 

Maintenance 

.4708236 

.6591531 

.2197177 

.4080471 

5 

Personnel 

Service 

Compliance 

Maintenance 

.5178575 

.4488098 

.3797622 

.5869052 

6 

Personnel 

Service 

Compliance 

Maintenance 

.1627577 

.7324096 

.1627577 

.4882731 

7 

Personnel 

Service 

Compliance 

Maintenance 

.4160030 

.733^790 

.2408438 

.3229497 

All 

Personnel 

Service 

Compliance 

Maintenance 

.3823280 

.6111227 

.3006694 

.4656031 

Consistency 

Index  (Avi 

Relative 

Perceived 

Weight 

Weight 

r£ 

.10 

— 

.90526 

.29 

— 

.42 

— 

.19 

— 

.25 

.25 

.98771 

• 33 

• 30 

.09 

.15 

.33 

.30 

.09 

.20 

.97228 

• 55 

• 35 

.10 

.20 

.26 

• 25 

• 25 

.20 

.87094 

• 50 

• 35 

.06 

.10 

.19 

.35 

.28 

.25 

.95828 

.21 

.25 

.15 

.20 

• 36 

• 30 

.03 

_ _ 

.82781 

.65 

-- 

.03 

-- 

.29 

— 

.20 

• 25 

.37335 

.62 

.28 

.06 

.22 

.12 

.25 

.18 

N/A 

.82683 

.4-5 

N/A 

.11 

N/A 

.26 

N/A 

age  R2) : 0.91366 


a significant  departure  from  the  stated  goals  of  such 
managers.  The  next  question  which  comes  to  mind  is:  How 
well  are  the  actual  (operative)  goals  of  these  executives 
understood  and  supported  by  their  subordinates  and  by  units 
subordinate  to  the  Command  headquarters? 

Results  of  the  Main  Data  Base  Analysis 

The  answer  to  the  above  question  lies  in  the  analysis 

of  responses  from  the  AFCS  middle  managers.  The  main  body 

of  the  computation  results  is  contained  in  Appendix  F.  This 

Appendix  shows  the  relative  weights  computed  for  each  group 

of  AFCS  managers,  the  beta-weights  for  each  regression,  the 
2 

regression  R and  residual  sum  of  squares  and,  finally,  the 
observed  F value  for  testing  goal  congruence  between  the 
groups  in  each  regression.  Table  4-3  presents  a summary  of 
analytical  results  for  middle  managers  similar  to  that  pro- 
vided earlier  for  senior  manager  results.  All  managerial 
groups  in  Table  4-3  and  subsequent  tables  have  been  referred 
to  by  numerical  designators.  Identification  of  the  groups 
is  considered  proprietary  information  and  is  not  considered 
necessary  to  understanding  of  this  report.  Should  the  read- 
er deem  it  essential  to  know  such  details,  he  should  contact 
AFCS/CA,  Richards -Gebaur  AF3,  Missouri  64030. 

A superficial  perusal  of  the  middle  management  analy- 
sis results  reveals  what  could  be  taken  as  striking  similari- 


ties between  the  seven  groups  of  managers.  For  example, 
four  of  the  seven  groups  weighted  the  Quality  of  Service 
factor  at  exactly  the  same  value  (0.49).  Similarly,  the 


Table  4-3.  Middle  Manager  Analysis  Results. 


Management 

Beta 

Relative 

_ 2 

Group 

Predictor 

Weight 

Weight 

R 

1 

Personnel 

•353^649 

.16 

.76104 

Service 

. 6o?684 6 

.49 

Compliance 

• 2792534- 

.10 

Maintenance 

.4346199 

.25 

2 

Personnel  . 

•3576236 

.18 

.72728 

Service  -• 

.5643008 

.44 

Compliance 

.3332323 

.15 

Maintenance 

.4121910 

.23 

3 

Personnel 

.3302284 

. 14 

.77375 

Service 

.6301606 

• 51 

Compliance 

.3140704 

.13 

Maintenance 

•4170775 

.22 

4 

Personnel 

.3616129 

.17 

.77501 

Service 

.6173115 

. 49 

Compliance 

.3252636 

. 14 

Maintenance 

.3967088 

.22 

5 

Personnel 

.3221150 

. 14 

.74185 

Service 

.6039127 

.49 

Compliance 

•3106724 

• 13 

Maintenance 

.4205441 

.24 

6 

Personnel 

.2904820 

.11 

.77735 

Service 

.6180874 

.49 

Compliance 

.3445501 

• 15 

Maintenance 

.4390004 

.25 

7 

Personnel 

.3044796 

.12 

.76711 

Service 

.6241570 

.51 

Compliance 

.3474526 

.16 

Maintenance 

.4040994- 

.21 

All  Sub- 

Personnel 

.3282215 

.14 

.75332 

jects 

Service 

.6095076 

.49 

Compliance 

.3212979 

. 14 

Maintenance 

.4193538 

.23 

Consistency  Index  (Average  R ):  0.89016 
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rank  ordering  of  the  four  performance  criteria  is  the  same 
for  six  of  the  seven  groups,  the  single  exception  being 
Group  6 managers,  who  placed  relatively  more  weight  on  the 
Compliance  factor  than  did  the  other  groups.  The  rank  or- 
dering in  all  groups  except  Group  6 was  as  follows: 

X2  (Quality  of  Service) 

X4  (Maintenance  Quality) 

Xl  (Personnel  Programs) 

X3  (Compliance) 

For  the  Group  6 managers,  the  Compliance  and  Personnel  cri- 
teria were  interchanged. 

2 

The  relatively  lower  R value  for  these  groups  compar- 

2 

ed  with  the  individual  R of  each  of  the  senior  managers  is 
readily  explainable.  Each  manager  applies  his  own  evalua- 
tion policy  with  a fairly  high  degree  of  consistency:  how- 
ever, when  a number  of  individuals  is  taken  as  a composite 
group,  the  apparent  decision,  consistency  is  decreased  due  to 
differences  in  individual  evaluation  policies.  This  is 

proven  by  averaging  the  individual  consistencies  of  all 

2 2 

respondents  and  comparing  this  averaged  R with  the  R of 

the  combined  regression  runs.  As  shown  in  Table  4-3,  the 

consistency  index  for  all  513  middle  manager  respondents 

was  0.89016.  The  R of  the  respondents  combined  as  a group 

was  O.75832.  These  findings  are  consistent  with  those  from 

the  senior  managers,  whose  group  R was  0. 82683  while  the 

2 

individual  averaged  R , or  consistency  index,  was  O.91366. 

This  first,  rather  cursory  glance  at  the  analysis 
findings  seems  to  suggest  a rather  marked,  Command-wide 
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homogeneity  in  the  decision  behavior  of  AFCS  managers;  how- 
ever, the  real  test  of  this  apparent  agreement  lies  in  a 
more  rigorous  comparison  of  the  managerial  groups. 

F-Test  Results 

Employing  the  test  detailed  in  Appendix  E,  appropriate 
groups  of  middle  management  unit  evaluations  were  compared 
to  determine  whether  any  such  groups  represented  decisions 
by  statistically  different  populations.  As  Appendix  E 
shows,  the  F-Test  value  is  calculated  using  the  residual 
sums  of  squares  of  all  regression  models  in  the  comparison. 

The  observed  F /alue  thus  calculated  is  compared  against  the 
appropriate  value  for  a standard  F distribution,  at  the 
chosen  level  of  significance,  to  decide  whether  or  not  to 
reject  the  null  hypothesis  that  all  compared  models  have 
statistically  similar  regression  coefficients.  Table  4-4 
shows  the  AFCS  management  groups  thus  compared,  the  number 
of  decisions  (N)  made  by  each  composite  group,  the  appropriate 
F values,  and  an  indication  of  whether  or  not  the  null 
hypothesis  was  rejected  in  each  case.  Figure  4-1  is  a 
more  graphic  portrayal  of  the  F-Test  results.  The  solid 
lines  in  this  figure  are  intended  to  indicate  connections 
between  statistically  similar  populations,  whereas  the 
broken  lines  between  groups  suggest  differences  in  the 
goals  of  the  groups  thus  connected. 

Although  there  are  more  rejections  of  the  null  hypo- 
thesis than  there  are  acceptances,  it  was  considered  pos- 
sible that  each  group  might  simply  be  optimizing  its  own 

48 


fJSt  j lj 


AFCS, 

1* 


Reject  Hq 

Do  not  re- 
ject H0 


Figure  4-1.  Graphic  Depiction  of  F-Test  Results. 

slightly  different  goals  to  the  same  general  effect.  That 
is,  while  the  goals  being  considered  in  the  managerial 
decision  process  might  he  somewhat  different  for  each  group, 
the  decisions  reached  might  still  reflect  the  general  policy 
of  the  Command  as  a whole.  To  test  this  hypothesis,  it  was 
necessary  to  establish  a Command  standard  for  evaluating 
individual  groups.  Accordingly,  it  was  reasoned  that  the 
average  weights  used  by  the  senior  executives  of  the  Command 
in  making  identical  decisions  could  be  used  as  comparative 
bases  for  determining  whether  the  decisions  of  AFCS  middle 
managers  represent  the  interests  of  the  Command.  This 
seems  natural  and  defensible,  since  the  senior  staff  of- 
ficers in  the  Command  headquarters  are  the  de  jure  policy- 
setters  for  AFCS. 
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Table  4-4.  F-Test  Results. 

Reject 


Comparison  Groups 

N 

£0 

F.05 

Ho' 

1/2/3/V5/6/7 

8208 

37.395 

1.57 

Yes 

1/2 

2112 

3-574 

2.21 

Yes 

1/3 

2224 

1.387 

2.21 

No 

1/4 

2048 

3.287 

2.21 

Yes 

1/5 

2752 

1.452 

2.21 

No 

1/6 

2480 

4.658 

2.21 

Yes 

1/7 

2192 

4.826 

2.21 

Yes 

2/3 

2096 

3.439 

2.21 

Yes 

2/4 

1920 

1.819 

2.21 

No 

2/5 

2624 

3-138 

2.21 

Yes 

3/4 

2032 

1.712 

2.21 

No 

3/6 

2464 

2.682 

2.21 

Yes 

4/7 

2000 

1.947 

2.21 

No 

5/6 

2992 

2.762 

2.21 

Yes 

5/7 

2704 

1.323 

2.21 

No 

6/7 

2432 

3. 518 

2.21 

Yes 

Further  Decision  Comparisons 
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The  Student  t-Test  described  by  Speigel  (1975.  p.  215) 
and  by  Freund  (1971,  pp.  317-318)  was  used  to  test  the  null 
hypothesis  that  the  l6  evaluations  made  by  each  of  the 
various  AFCS  management  groups  represented  the  same  decisions 
as  like  evaluations  made  by  the  senior  AFCS  staff.  This 
hypothesis,  of  course,  was  tested  against  the  alternate 
hypothesis  that  the  l6  decisions  of  each  group  were  not  the 
same  as  those  made  by  the  senior  staff;  thus,  a two-tailed 
test  was  indicated,  and  a 0.05  level  of  significance  was 
used.  The  formula  for  calculating  the  observed  t value  for 
each  decision  is  as  follows: 


where  is  the  mean  of  hypothetical  unit  evaluation  i 

(i  = 1,2 16)  calculated  for  all  members  of  a given  group, 

c^  is  the  mean  of  the  same  evaluation  calculated  from  the 
seven  senior  manager  responses,  s^  is  the  standard  deviation 
of  the  group  evaluation  being  tested,  and  n is  the  number  of 
respondents  in  the  group  sample. 

It  was  realized  that  there  are  philosophical  arguments 
against  this  approach,  since  obviously  all  seven  senior 
managers  did  not  precisely  agree  as  to  the  goals  of  AFCS. 

This  means,  of  course,  that  there  is  a statistical  distribu- 
tion involved  in  the  senior  manager  decisions.  Nonetheless, 
it  was  reasoned  that  these  seven  senior  executives  consti- 
tute a substantial  majority  of  the  policy-setting  body  of  the 
Command;  therefore,  the  mean  of  each  of  their  l6  evaluations 
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was  viewed  as  a population  parameter,  rather  than  an  esti- 
mate, against  which  to  compare  the  means  of  like  decisions 
by  any  given  subordinate  group . 


Appendix  G contains  the  actual  t-Test  scores  calculated 
for  each  middle  management  group;  however,  the  comparison 
results  have  been  summarized  in  Table  4-5.  This  Table  shows 
that  the  Command  senior  executives  and  various  middle  manage- 
ment groups  agree  on  as  few  as  8 to  as  many  as  12  of  the  16 
evaluations . 

Table  4-5.  t-Test  Results 


Middle 

Management 

Group 

Number  of 
Decisions 
Agreeing  With 
Senior  Staff 

Percentage 
Of  Agreement 

1 

12 

75? 

2 

11 

68 . 75? 

3 

12 

75?° 

4 

3 

50? 

5 

12 

75? 

6 

10 

62.5? 

7 

11 

68 . 5? 

Sensitivity  of  the  Regression  Model 

As  a final  note  on  the  experimental  results,  the 
sensitivity  of  the  linear  regression  model  was  tested  by 
varying  the  values  of  a single  evaluation  in  one  model 
through  several  regression  runs.  It  was  found  that  this 
resulted  in  extreme  differences  in  the  multiple  correlation 
coefficient.  For  example,  the  regression  intially  produced 

p 

an  R value  of  0.82781  with  decision  number  12  being  valued 
at  1.0.  When  this  decision  value  was  changed  full  range 
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to  8.0,  the  regression  was  reduced  to  0.21579*  Inter- 
mediate values  in  the  range  of  the  same  decision  resulted  in 

p 

corresponding  R variances;  hence,  the  model  is  extremely 
sensitive  to  changes  in  the  rating  policy  of  the  evaluator. 


Chapter  V 


SUMMARY  AND  CONCLUSIONS 


a ummar^ 


The  Air  Force  Communications  Service  is  a large, 
widely  dispersed  Air  Force  support  organization  with  complex 
intracommand  managerial  interfaces.  Such  an  organization  in- 
evitably has  the  potential  for  difficulty  in  communicating 
the  goals  and  policies  of  its  highest  executives  through  its 
several  managerial  echelons  to  those  who  do  the  work  of  the 
Command.  This  research  effort  was  initiated  as  an  attempt 
to  determine  how  successful  the  communication  of  Command 
goals  is  within  AFCS . In  other  words,  the  research  was 
intended  to  address  the  specific  question  of  whether  or  not 
goal  congruence  exists  among  the  various  AFCS  management  levels 
The  study  methodology  is  a logical  follow-on  to  the 
work  of  many  contemporary  social  scientists  who  have  used 
linear  regression  models  to  capture  individual  policy  and 
predict  human  judgement.  In  this  study,  however,  the  linear 
model  is  used  as  an  analytical  instrument  rather  than  a pre- 
dictive one,  in  a manner  similar  to  that  used  by  Harrell 
(1975)-  The  technique  involves  the  use  of  controlled  pre- 
dictor variables  as  information  cues  for  the  experimental 
subject  to  employ  in  making  decisions.  In  this  study  the 
predictors  were  cues  related  to  four  familiar  operative 
goals  of  AFCS,  as  perceived  by  the  researcher  and  agreed 
upon  by  the  AFCS  headquarters  staff  (See  Appendix  A) . 

Each  participant  in  the  field  experiment  was  asked  to  use 


the  cues  provided  him  to  evaluate  the  performance  of  l6 
hypothetical  AFCS  squadrons.  The  evaluations  thus  provided 
by  513  AFCS  middle  managers,  along  with  responses  from 
seven  of  eight  senior  AFCS  staff  officers,  were  subjected 
to  step-wise  multivariate  regression  analysis,  the  data 
being  grouped  by  management  level  and  by  unit  within  each 
level . 

The  results  of  the  regression  analysis,  which  provided 
eight  separate  linear  regression  models,  were  then  compared 
to  determine  whether  or  not  the  unit  evaluations  of  the  var- 
ious management  groups  were  statistically  different.  It 
was  found  that  goal  congruence  existed  in  five  of  the  fif- 
teen comparisons  made.  In  comparisons  between  command 
levels,  agreement  was  found  between  one  of  the  three  Areas 
and  the  Command  headquarters  and  between  the  operating  units 
of  another  Area  and  AFCS  headquarters.  Similarly,  the  op- 
erating units  of  only  one  Area  were  found  to  be  in  agree- 
ment with  the  Area  headquarters. 

To  analyze  the  significance  of  these  differences  be- 
tween command  levels,  a further  test  was  made  to  determine 
whether  the  various  management  groups  might  be  emphasizing 

i 

different  goals,  but  still  making  decisions  compatible  with 
those  of  the  senior  executives  of  the  Command.  It  was 
found  that  the  decisions  of  the  subordinate  groups  of 
managers  were  compatible  with  those  of  the  Command's  senior 
managers  in  50$  to  75$  of  the  decisions  made. 

A rather  surprising  outcome  of  the  analysis  was  the 
relatively  greater  importance  placed  on  Compliance  by  the 
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operating  units  vis-a-vis  the  Command  headquarters.  The 
intermediate  headquarters  staffs  (Areas)  also  emphasized 
Compliance  more  than  the  Command  headquarters,  but  less 
than  the  operating  units. 

One  analytical  result  which  AFCS  should  find  very 
satisfying  is  the  pervasiveness  of  the  Service  goal  among 
its  managers.  While  it  is  one  thing  to  dictate  the  goals  of 
an  organization,  it  is  quite  another  thing  to  find  a single 
goal  to  be  as  powerfully  operative  throughout  an  organiza- 
tion as  is  the  goal  of  service  to  the  customer  among  AFCS 
middle  managers.  In  every  group  studied  the  Service  goal 
was  by  far  the  most  important  consideration  in  the  manage- 
ment decisions  made. 

Conclusions 

The  following  conclusions  were  arrived  at  after 
analysis  of  the  experimental  data: 

a.  There  are  significant  differences  in  the  goals 
emphasized  by  AFCS  middle  managers  at  different  levels  of 
that  segment  of  the  Command  which  participated  in  this 
study.  As  a result  of  these  differences,  it  appears  that 
the  various  groups  of  AFCS  middle  managers  would  make 
decisions  that  were  different  from  those  reached  by  the 
senior  managers  of  the  Command  about  25#  to  50%  of  the  time. 

b.  The  linear  regression  model  is  an  effective 
analytical  device  for  organizational  studies  involving  goals 
and  decision  making.  It  is  believed,  on  the  basis  of  the 
author's  literature  review,  that  this  is  the  first  research 
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effort  where  the  techniques  of  linear  modeling  have  been 
used  to  capture  the  decision  policies  of  a multilevel  or- 
ganization, or  to  test  for  goal  congruence  between  organ- 
izational levels.  It  appears  that  this  useful  research 
technique  might  have  wide  application  in  many  types  of  or- 
ganizational  analysis. 

Further  Implications  of  the  Results 

It  seems  likely  that  the  degree  of  goal  congruence  be- 
tween AFCS  units  participating  in  this  study  could  be  im- 
proved by  the  initiation  of  a management  information 
system  designed  to  provide  a more  efficient  downward  flow 
of  policy  information.  AFCSR  100-1?  (1975'.  AFCSR  124-1 
(1974),  AFM  65-110  (1975).  and  AFM  65-265(1973).  among 
others,  all  specify  information  to  be  reported  by  the 
operating  units  to  their  respective  Areas,  and  likewise  by 
the  Areas  to  AFCS  headquarters,  on  a rather  immediate  basis; 
however,  no  references  were  found  concerning  similarly  rapid 
downward  or  lateral  information  flow.  Interviews  with  AFCS 
managers  indicated  that  avenues  for  communicating  new  com- 
mand policy  appear  to  be  limited  to  inspection  reports, 
staff  visit  reports,  infrequent  conferences,  and  occasion- 
al visits  by  Area/AFCS  headquarters  commanders.  The  dis- 
semination of  command  policy  in  general,  of  course,  is  done 
through  the  media  of  regulations,  manuals,  and  other  written 
guidance.  It  seems  likely  that  some  more  immediate  form 
of  two-way  management  information  exchange  would  improve 
goal  congruence  between  command  levels. 
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The  importance  of  Compliance  at  the  lower  levels  of 
the  command  suggests  that  either  there  is  a belief  at  the 
operating  units  that  compliance  is  extremely  important  to 


higher  AFCS  levels  or  there  is  a perceivedneed  to  rely 
heavily  upon  higher  command  directives  for  guidance.  Past 
experience  of  the  researcher  and  discussion  with  others 
having  AFCS  command  experience  tends  to  negate  the  latter 
reasoning.  With  regard  to  the  former  rationale,  discussions 
with  the  AFCS  Inspector  General  have  led  to  repeated  assur- 
ances that  Command  emphasis  on  Compliance  has  dramatically 
decreased  in  the  past  two  years.  Experimental  findings 
from  among  respondents  at  the  Command  headquarters  tend  to 
support  this  philosophy,  yet  the  operating  units  apparently 
do  not  perceive  a de-emphasis  of  Compliance  by  the  various 
headquarters  staffs.  This  paradox  seems  to  suggest  once 
again  that  the  operative  goals  of  the  higher  echelons  are 
not  being  adequately  communicated  to  the  operating  units. 

Despite  the  apparent  lack  of  goal  congruence  within 
the  tested  AFCS  population,  there  is  a high  degree  of  con- 
sistency in  the  decisions  of  AFCS  managers.  This  implies 
that  AFCS  managers  are  generally  capable  of  incorporating 
their  individual  policies  into  the  decision  process  in  an 
unambiguous  way,  suggesting  in  turn  a high  degree  cf  dis- 
cipline and  dedication.  This  tends  to  support  once  more  the 
earlier  implication  that  greater  goal  congruence  within  the 
Command  is  possible. 

Finally,  the  hierarchical  order  in  which  the  four 
performance  criteria  were  entered  into  the  step-wise 
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regressions  was  generally  consistent  throughout  the  study. 
This  order  being  directly  associated  with  the  relative  im- 
portance placed  upon  the  criteria  by  the  decision  makers, 
it  seems  probable  that,  while  different  relative  weights 
are  assigned  to  the  criteria  by  the  various  groups,  there 
is  general  agreement  on  the  viability  of  the  goals  chosen 
for  this  study.  That  is,  there  is  little  doubt  that  the 
four  selected  goals  are  operative  within  the  Command. 

Suggestions  for  Further  Research 

It  would  be  interesting  to  extend  this  study  into 
other  AFCS  units  to  determine  whether  or  not  the  results 
could  be  generalized  to  the  entire  Command.  In  particular, 
since  some  of  the  greatest  differences  in  operative  goals 
appear  to  occur  between  the  Command  headquarters  and  units 
having  some  direct  affiliation  with  other  commands,  it 
would  probably  be  fruitful  to  analyze  the  remaining  three 
Areas  to  determine  if  this  apparent  trend  might  be  real. 

Another  interesting  possibility  for  further  research 
exists  in  the  extension  of  Harrell's  (1975)  techniques  to 
AFCS  to  test  for  the  effects  of  policy  cues  and  feedback 
in  the  experiment.  These  factors  can  provide  much  meaning- 
ful data  on  the  probable  effectiveness  of  management  in- 
formation systems,  and  might  suggest  a form  of  information 
system  best  suited  to  the  Command. 

Other  Comments 

There  are  several  important  points  to  be  made  relative 
to  the  above  conclusions  and  implications,  as  follows: 
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a.  No  attempt  has  been  made  in  this  study  to  state 
just  how  important  goal  congruence  might  be  to  AFCS . The 
literature  tends  to  presume  that  goal  congruence  is  a de- 
sirable state  to  be  maintained  within  an  organization.  If 
the  organization  supports  this  assertion,  then  the  lack  of 
goal  congruence  would  probably  be  considered  a state  which 
some  attempt  might  be  made  to  correct. 

b.  So  little  work  has  been  done  in  the  area  of  testing 
organizations  for  goal  congruence  that  it  would  not  be 
possible  or  desirable  at  this  point  to  imply  in  any  way 
that  there  is  more  or  less  goal  congruence  in  AFCS  than  in 
any  other  organization. 

c.  No  attempt  was  made  in  this  study  to  say  what  goals 
of  AFCS  should  be  emphasized.  In  fact,  no  attempt  has 
been  made  to  say  which  of  the  four  operative  goals  chosen 
for  the  study  itself  should  by  emphasized  by  AFCS  managers. 

Related  Study 

Another  study  is  presently  being  conducted  to  determine 
whether  or  not  some  of  the  differences  in  goals  found  in  this 
research  effort  might  be  related  to  the  demographics  col- 
lected with  the  experimental  exercise.  This  second  study 
will  rely  heavily  on  the  Automatic  Interaction  Detection 
(AID)  algorithm  to  compare  the  unit  evaluations  against  such 
factors  as  the  subject’s  grade,  time  in  assignment,  previous 
assignment,  military/civilian  status,  education,  etc.  It  is 
possible  that  some  improvement  in  the  predictive  power  of 
the  linear  models  can  be  brought  about  by  such  demographic 
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groupings;  however,  it  should  be  understood  that  the  models 
derived  in  this  study  have  been  demostrated  to  be  both  very 
sensitive  and  quite  powerful  for  the  applications  described 
here . 
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ATTN  O* 


DEPARTMENT  of  the  AIR  force 

H£AOC'JAST£SS  A =5  eOPZZ  COMvu'^CAT'CNi  5;5VC; 
^ CHAnDS’C’EsAOR  a r jACE  9-s£  '■’  3 3 C v- = SAC3C 


susjscT:  Research  Proposal 


„ Lt  Col  Carl  G.  O'Berry 
4111  Silver  Oak  Street 
Dayton,  OH  45424 

1.  The  research  proposal  shown  in  your  letter  of 

1 Dec  1976  dealing  with  the  goal  congruence  between 
AFCS  Headquarters  and  the  six  Areas  and  between  the 
Areas  and  the  Operating  Units  has  been  reviewed,  and 
the  following  comments  are  offered: 

a.  We  agree  in  principle  with  the  proposal;  the  four 
decision  criteria  (personnel,  service  quality,  compliance 
and  maintenance  quality)  appear  realistic  and  independent 
Moreover,  they  represent  valid  indicators  which  relate 

to  the  decision  making  policies  associated  with  this 
Command . 

b.  The  data  collection  method  for  the  proposed 
research  should  be  coordinated  with  AFCS  prior  to  actual 
implementation.  The  anticipated  requirements  for  AFCS 
resources  must  be  defined  prior  to  final  approval. 

2.  Point  of  contact  at  this  Headquarters  is  Mr.  Thomas 
YiumJ^AUTOVON  465-3631,  or  (816)  348-3631. 


4111  Silver  Oak  Street 
Dayton,  Ohio  45424 
1 December  1976 

Mr.  Thomas  Yium 
AFCS/OA 

Richards-Gebaur  AFB,  MO  64030 
Dear  Toms 

Per  our  recent  telephone  discussion  the  attached 
research  proposal  is  forwarded  for  your  consideration.  I 
will  be  free  during  the  week  of  20  December  and  can  travel 
to  Richards-Gebaur  for  further  discussion  of  the  proposal 
should  you  so  desire.  It  may  prove  mutually  advantageous 
for  us  to  get  together  to  insure  that  we  are  in  complete 
agreement  on  decision  criteria  and  research  objectives.  I 
would  appreciate  hearing  from  you  as  soon  as  you  have  had  the 
opportunity  to  look  the  proposal  over.  I think  we  can  do 
a good  job  for  you. 

Regards, 


Carl  G.  O' Berry 
Lt.  Col.,  USAF 
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A Research  Proposal 
Submitted  to 

Air  Force  Communications  Service/OA 

by 

Carl  G.  O' Berry,  Lt.  Col.,  USAF 


Background 

1.  There  are  two  unanswered  questions  pertaining  to 
the  Air  Force  Communications  Service  management  structure 
which  seem  to  occur  to  nearly  every  unit  and  Area  commander 
at  one  time  or  another.  These  same  questions  also  seem  to 
recur  regularly  within  the  staff  agencies  of  both  the  Areas 
and  the  command  headquarters.  A general  form  taken  by  these 
questions  might  be  stated  as  follows: 

a.  To  what  degree  does  the  next  higher  command 
level  need  to  be  involved  with  the  day-to-day  activities  of 
the  operating  unit/Area? 

b.  How  much  decision-making  autonomy  should  the 
Headquarters  grant  to  the  Areas  and  the  Areas  to  the 
operating  units? 

2.  Both  the  above  questions  appear  related  to  the  need 
(or  lack  of  need)  for  some  type  of  management  information 
system.  There  has  been  an  almost  continuous  attempt  since 
the  early  1960's  to  satisfy  varying  requirements  for  infor- 
mation at  all  AFCS  command  levels.  AFCSR  100-17  and  AFC3R 
125-1  both  detail  reporting  requirements  intended  to  keep 
the  command  informed  of  activities  at  the  units.  There  are 
also  several  other  reporting  systems  which  provide  informa- 
tion to  the  command,  such  as  those  prescribed  in  AFM  65-110 


66 


and  AFM  6 5-265.  The  questions  persist,  nonetheless,  and  it 
appears  that  it  might  prove  fruitful  to  take  a new  approach 
to  these  questions. 

3.  If  it  can  be  established  that  the  two  questions  cited 
above  actually  stem  from  a single  basic  question  related  to 
management  of  the  command,  then  the  picture  can  be  somewhat 
simplified.  This  kind  of  simplification  is  suggested  by  a 
technique  known  in  management  accounting  as  goal  congruence 
testing.  Specifically,  the  need  for  management  information 
systems  can  be  tied  to  the  one  very  precise  question:  Does 
goal  congruence  exist  between  the  various  echelons  of  the 
organization? 

4.  There  are  a number  of  techniques  for  analyzing  an 

organization  for  goal  congruence.  Extensive  work  is  current- 
ly being  done  in  the  area  of  statistical  modeling  for  assess- 
ment of  personal  and  organizational  behavior,  and  this  work 
is  adaptable  to  the  AFCS  community.  For  example,  multivar- 
iate regression  analysis  is  a familiar  statistical  means  for 
establishing  relationships  in  behavior  patterns.  Re- 

gression analysis,  coupled  with  the  use  of  recently  develop- 
ed policy-capturing  techniques,  can  be  used  to  predict  the 
behavior  of  individuals  and  groups  with  considerable  accuracy. 

5.  The  particular  research  problem  proposed  for  your 
consideration  is  the  following:  Does  goal  congruence  exist 
between  AFCS  Headquarters  and  the  six  Areas,  and  between  the 
Areas  and  the  operating  ’units?  Applying  statistical  analysis 
techniques  to  this  question  may  reveal  possibilities  for 
application  of  innovative  management  ideas.  By  carefully 


choosing  the  analytical  techniques,  insuring  the  gathering 
of  unbiased  data  through  random  selection  processes  and  draw- 
ing the  correct  conclusions  from  the  research,  it  is  entirely 
possible  to  derive  information  about  the  command  which  may  be 
very  far-reaching  in  impact.  Until  something  is  known  about 
the  nature  of  the  relationships  between  the  various  command 
echelons  it  is  difficult  to  establish  either  an  optimal 
management  information  system  or  any  meaningful  measure  of 
unit  efficiency.  These  relationships  are  complex  management 
equations  which  are  influenced  by  many  internal  and  external 
factors,  some  of  which  may  wield  greater  power  than  is  present- 
ly known.  Therefore,  the  answer  to  the  basic  question  of  goal 
congruence  appears  to  be  essential  for  meaningful  evaluation 
of  the  command  management  system  as  a whole. 

Objectives  and  Purpose  of  Research 

1.  To  insure  that  the  question  of  goal  congruence  can 
be  addressed  for  the  entire  command  while  at  the  same  time 
limiting  the  research  to  managable  proportions,  it  is  pro- 
posed that  AFCS  sponsor  two  separate,  but  related,  simultan- 
eous research  endeavors.  One  of  these  would  be  directed  at 
the  MAJCOM/Area  interface  and  the  second  would  address  the 
Area/  operating  unit  relationship.  Contact  has  been  made 
with  a second  AFIT  student,  Captain  Jacob  P.  Miller,  who  is 
interested  in  pursuing  research  in  this  area,  and  with  a 
member  of  the  AFIT  faculty  who  has  indicated  not  only  in- 
terest in  the  research  but  willingness  to  serve  as  a con- 
sultant to  AFCS  for  this  and  any  follow-on  effort  which  may 
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result.  The  faculty  member,  Lt  Col  Adrian  Harrell,  holds  a 
Ph.D.  in  accounting  and  has  conducted  a study  similar  to  that 
proposed  here  as  a basis  for  his  doctoral  dissertation. 


2.  The  product  of  the  research  herein  proposed  would 
be  a potentially  powerful  management  aid  to  AFCS,  The  re- 
search could  also  be  used  as  the  basis  for  comparative  anal- 
ysis against  other  studies  or  as  the  starting  point  for  further 
scientific  management  study  within  the  command.  The  research 
should  prove  to  have  considerable  utility  to  the  command  in 
developing  and  producing  an  effective  management  information 
system  and  for  other  management  applications. 

Methodology 

1.  The  data  collection  method  for  the  proposed  research 
has  not  yet  been  fully  developed,  since  there  are  some  points 
which  must  be  jointly  decided  upon  by  the  command  and  those 
conducting  the  research.  The  suggested  approach  involves  a 
field  experiment  to  collect  data  related  to  the  decision- 
making policies  of  middle  management  personnel  throughout  the 
command.  By  selecting  appropriate  decision  criteria  and 
testing  randomly  selected  personnel  at  the  headquarters,  Areas 
and  operating  units  it  is  hoped  that  a basis  may  be  extablish- 
ed  for  determining  the  degree  to  which  these  personnel  are 
in  agreement  in  interpreting  policy  and  to  determine  the 
respective  weights  that  various  levels  place  on  the  pre- 
selected decision  criteria.  Specifically,  each  participant 
would  be  asked  to  evaluate  a number  of  hypothetical  AFCS 
units  on  the  basis  of  whether  or  not  the  decision  criteria 
were  satisfied  for  each  unit.  The  proposed  decision  criteria 
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should  be  selected  on  the  basis  of  relevance,  independence, 
and  uniformity  of  interpretation  (lack  of  ambiguity).  The 
following  criteria  appear  to  meet  these  requirements:  personnel 
factors,  quality  of  service  factors,  compliance  factors,  and 
maintenance  quality  factors. 

a.  Personnel.  Includes  factors  such  as  morale/ 
welfare  programs,  unit  disciplinary  rates,  IG  complaint  rates, 
retention  rates. 

b.  Service  Quality.  Factors  related  to  the  service 
provided  to  the  customer  by  the  unit  being  evaluated.  Meas- 
ured by  means  of  speed  of  service,  message  handling  time, 
operator  courtesy,  air  traffic  control  quality,  responsive 
ness  to  new  requirements,  etc. 

c.  Compliance,  Factors  related  to  how  closely  the 
unit  follows  AF,  AFCS,  and  Area  directives  as  determined  by 
IG  reports,  ATC  analysis  reports,  staff  visit  reports. 

d.  Maintenance  Quality.  Factors  indicative  of 
maintenance  effectiveness  and  efficiency,  such  as  ready  rates, 
uptime  rates,  etc. 

2.  Assuming  agreemt.it  on  these  four  decision  criteria, 
each  participant  would  evaluate  a total  of  16  hypothetical 
units,  rating  each  unit  on  a scale  of  1 through  8 (highly  un- 
satisfactory to  highly  satisfactory)  based  on  preassigned 
grades  of  satisfactory  or  unsatisfactory  for  each  of  the 
decision  criteria.  Sixteen  is  the  number  of  units  chosen  for 
the  test  in  order  to  cover  all  possible  combinations  of 
grading  (3  or  U)  of  all  four  decision  criteria  (see  figure 
2).  The  assignment  of  the  grades  of  3 or  U to  the  criteria 
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would  be  done  in  a random  fashion  to  avoid  possible  sequential 
biases  and  the  evaluated  units  would  be  arranged  in  random 


order  to  discourage  "gaming."  The  evaluation  sheets  would 
appear  essentially  as  shown  in  figure  1 . The  data  thus 
collected  would  be  subjected  to  the  appropriate  statistical 
analysis  to  determine  what,  if  any,  relationship  there  is 
between  decisions  made  at  various  levels  of  the  command. 

3.  In  view  of  the  complexity  of  the  AFCS  organization 
and  command  lines  it  would  be  surprising  if  the  proposed 
field  experiment  did  not  yield  some  highly  interesting  re- 
sults. It  is  entirely  possible  that  external  factors  could 
have  significant  impact  on  unit  response  to  AFCS/  Area 
policy  and  direction.  Conversely,  it  is  also  possible  that 
there  is  a very  close  coincidence  of  goals  between  the  various 
echelons.  The  latter  case  could  indicate  that  greater 
autonomy  than  is  presently  thought  possible  would  produce 
greater  efficiency  and  better  service  to  AFCS  customers.  In 
either  case,  the  field  experiment  should  be  of  significant 
benefit  to  AFCS  and  the  cost  of  this  effort  to  the  command 
would  be  very  small  compared  to  the  cost  for  use  of  a pro- 
fessional consulting  service. 

4.  If  the  command  accepts  sponsorship  of  the  proposed 
research  project,  the  following  tenative  schedule  would  apply: 

a.  Finalization  of  field  experiment  brochure:  Jan  77 

b.  Completion  of  literature  survey:  May  77 

c.  Conduct  of  field  experiments:  1-30  Jun  77 

d.  Defense  of  theses:  1 Aug  77 

e.  Final  thesis  to  AFCS:  NLT  1 Sep  77 

Note:  This  schedule  is  prolonged  due  to  an  unexpected 


71 


change  in  AFIT  policy  regarding  its  treatment  of  the  indepen- 
ent  study  portion  of  the  degree  program.  It  was  originally 
planned  that  the  field  experiment  could  be  conducted  in  the 
late  March  - early  April  time  period;  however,  the  new  school 
policy  is  to  extend  the  independent  study  phase  over  two 
academic  quarters  instead  of  one,  thus  preventing  completion 
of  the  proposed  research  by  June  1977  as  originally  indicated. 
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2.  For  further  information  concerning  this  proposal, 
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contact  Lt  Col  Carl  G.  O'Berry,  (513)  236-4376,  or  Lt  Col 
Adrian  Harrell,  Wright-Patterson  AFB  extension  52549. 


FIGURE  I 


Performance  Evaluation  of  Squadron  Number 1 

S=Satisfactory  U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated S 


2.  The  quality  of  service  provided  by  this  squadron 
is  rated.  .....  • • 


3.  Compliance  with  AF/AFCS/Area  directives  in  this 
squadron  is  rated  as 


A.  The  quality  of  maintenance  in  this  squadron  is 
considered  to  be 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly 

Unsatisfactory  Unsatisfactory 

Highly 

Satisfactory  Satisfactory 

- — 

APPENDIX  B 


DATA  COLLECTION  INSTRUMENT 

A copy  of  the  data  collection  instrument  for  the 
field  experiment  is  presented  in  this  Appendix.  In  addi- 
tion to  the  basic  instrument , a copy  of  the  cover  letter 
signed  by  the  ARCS  Chief  of  Staff  is  included,  as  is  an 
example  of  the  letters  used  to  forward  the  experimental 
exercise  to  each  of  the  eight  senior  ARCS  executives.  In 
addition  to  completing  the  basic  evaluation  exercise,  each 
senior  executive  was  ashed  to  specify  the  weights  he  felt 
should  be  applied  to  each  performance  criterion,  in  the 
manner  shown  on  the  page  following  the  example  explanatory 
letter. 

During  the  exercise,  the  subjects  were  asked  to  rate 
the  performance  of  a number  of  hypothetical  ARCS  communica- 
tions squadrons,  using  the  information  provided  as  to  wheth- 
er or  not  the  four  performance  criteria  were  satisfied. 

Since  each  criterion  was  categorized  as  being  either  met 

(satisfactory)  or  not  met  (unsatisfactory),  and  four  per- 
il 

formance  criteria  were  used,  there  were  2 = 16  distinct 

combinations  of  these  criteria.  In  this  experiment,  all  1 6 
possible  cases  were  presented  to  the  subject  for  his 
evaluation. 
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1.  The  attached  exercise  is  designed  to  investigate 
certain  aspects  of  the  decision-making  behavior  of 
AFCS  managers.  Request  your  participation  in  the 
exercise  by  rating  the  performance  of  sixteen 
hypothetical  Type  I communications  squadrons  based 
upon  whether  each  unit  is  rated  satisfactory  or 
unsatisfactory  with  respect  to  four  different  perform- 
ance criteria.  The  data  gathered  will  be  analyzed 
statistically  and  summarized  in  a thesis  to  be 
prepared  by  Air  Force  officers  studying  at  the  Air  Force 
Institute  of  Technology.  This  project  is  being  performed 
under  the  sponsorship  of  HQ  AFCS  and  has  full  command 
support . 

Z.  You  will  not  be  identified  with  the  data  in  the 
final  report  and  your  individual  responses  in  this 
exercise  will  not  be  made  known  to  anyone  except  the 
officers  conducting  the  study  at  AFIT.  If  you  wish, 
an  analysis  of  your  decision-making  performance  will 
be  provided  to  you  at  a later  date.  If  you  would  like 
to  receive  this  analysis,  so  indicate  on  page  3 of 
this  booklet.  The  analysis  will  be  sent  directly 
to  you  by  the  AFIT  students  conducting  the  study. 

3.  Please  follow  the  instructions  for  this  exercise 
carefully.  Your  support  can  help  to  increase  our 
understanding  of  AFCS  decision  dynamics  and  may  aid  in 
finding  areas  for  improvement  in  management  of  the 
command.  If  you  received  this  booklet  by  mail,  use  the 
enclosed  self-addressed  envelop  to  return  it  promptly 
to  AFIT/HNS.  Thank  you  for  your  cooperation. 


FOR  THE  COMMANDER 


1 Atc'n 

Decision  Analysis  Exercise 


DECISION  ANALYSIS  EXERCISE 


SPRING  1977 

AIR  FORCE  COMMUNICATIONS  SERVICE 
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NOTICE 


This  booklet  will  be  returned  directly  to  AFIT/ENS . All 
personal  data  will  be  removed  from  the  exercise  and  a numeric 
identifier  will  be  attached  to  each  booklet  returned.  No 
personal  data  will  be  forwarded  to  AEC5 , nor  will  any  such 
data  be  included  in  the  research  report.  Collection  of  the 
above  data  is  necessary  to  validate  the  authenticity  of  the 
exercise  results  and  to  aid  in  statistical  analysis.  No 
permanent  record  of  personal  data  will  be  retained  after  the 
analysis  is  completed. 
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BIOGRAPHICAL  DATA 


1 


Please  circle  or  write  in  the  appropriate  response. 


1.  Are  you  a military  officer  or  a civilian? 

A.  Military  officer 

B.  Civilian 


2.  Please  indicate  your  grade. 


A. 

0-1 

F . 

GS-9 

B. 

0-2 

G. 

GS-10 

C. 

0-3 

H. 

G3-11 

D. 

0-4 

I. 

GS-12 

E. 

0-5 

J. 

GS-13 

3.  Military  officers  please  indicate  your  date  of  rank. 
(Indicate  year/month,  e.g.s  March  1974  3 7403) 


4.  What  is  your  present  unit? 

A.  Hq  AFCS 
B • N G A 

C.  3ACCA 

D . 3CA 

E.  TCA 

P.  Group/Squadron,  (Indicate  unit  number,  e.g.s  2046, 
1398) 


What  is  your  office  symbol? 


6. 


What  year/month  were  you  assigned  to  your  present  unit? 
(Indicate  as  above,  e.g.s  October  1975  3 7510) 


7.  Have  you  attended  a resident  PME  course  other  than  SOS 
or  the  Communications-Electronics  Staff  School? 

A . Yes 

B.  No 
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8.  Have  you  attended  the  Communications-Electronics  Staff 
School? 

A.  Yes 

B.  No 

9.  .tfhat  was  your  previous  assignment? 

A, 


iviAJ  C 0M 

B.  Unit 

Level 

(1) 

USAF/JCS/DGD 

(1) 

Hq 

(2) 

AD  COM 

(2) 

Area/NAF 

(3) 

AECS 

(3) 

Group/S q 

(4) 

mAC 

(4) 

Other 

(5) 

SAC 

(6) 

TAC 

(7) 

USAFSS 

(8) 

Other 

10.  jv'hat  is  your  highest  level  of  education? 


A. 

3. 

C. 

D. 

1? 


Less  than  a bachelor's  degree 
Bachelor's  degree 

Graduate  work  beyond  a bachelor  degree 
Piaster's  degree 

Postgraduate  work  beyond  a master's  degree 
Doctorate  degree 


11.  Afhat  was/is  your  most  recent  major  field  of  study? 


A.  Engineering 

B.  Science/mathematics 

C . Management 

D.  Other 


12.  Do  you  wish  to  receive  an  analysis  of  your  performance 
on  this  exercise? 

A.  Yes 
3.  No 

13.  If  you  indicated  "Yes"  in  question  12,  please  print  your 
name  here. 


INSTRUCTIONS 


In  this  exercise  you  are  requested  to  rate  the  performance 
of  sixteen  hypothetical  AFCS  Type  I communications  squadrons. 
The  performance  criteria  upon  which  you  will  base  your  eval- 
uation of  each  squadron  consist  of  the  following: 


1.  Personnel  Programs.  Indicates  the  success  of  the  unit 
in  terms  of  morale  and  welfare  programs,  disciplinary  rate, 

IG  complaint  rate,  retention  rate,  etc. 


2.  Quality  of  Service.  Indicates  unit  success  in  factors 
related  to  the  service  provided  to  the  customer.  Includes 
such  factors  as  speed  of  service,  message  handling  time,  oper- 
ator courtesy,  accuracy,  air  traffic  control  quality,  and  re- 
sponsiveness to  new  customer  requirements. 

3.  Compliance . Indicates  how  closely  the  unit  being  rated 
follows  Air  Force,  AFCS,  and  Area  directives,  as  determined  by 
IG  reports,  Air  Traffic  Control  Analysis  reports,  and  staff 
visit  reports. 

Maintenance  Quality.  Indicates  maintenance  effective- 
ness and  efficiency  as  reflected  by  maintenance  analysis  re- 
ports, operational  ready  rates,  uptime  rates,  etc. 

These  four  performance  criteria  are  to  be  viewed  as  independent 
of  one  another.  A rating  of  satisfactory  or  unsatisfactory 
for  any  single  criterion  is  not  related  to,  and  does  not  in- 
fluence, the  ratings  of  the  other  criteria.  You  are  to  use 
only  the  information  provided  for  each  squadron  to  rate  the 
overall  performance  of  that  unit  as  one  of  the  following: 

1.  Highly  Satisfactory.  Minor  deficiencies  may  exist. 

Some  minor  corrective  action  may  be  required. 

2.  Satisfactory.  A few  significant  deficiencies  may 
exist.  Some  monitoring  by  the  Area  staff  might  be  indicated 
in  certain  areas . 

3.  Unsatisfactory.  Some  major  deficiencies  exist. 

Limited  on-site  Area  staff  assistance  may  be  required.  The 
replacement  of  at  least  one  key  unit  manager  might  be  con- 
sidered. 

4.  Highly  Unsatisfactory.  Many  major  deficiencies  exist. 
Immediate  corrective  action  and  extensive  on-site  Area  staff 
assistance  are  required.  Replacement  of  at  least  one  key 
unit  manager  is  recommended. 

Refer  to  the  example  on  the  following  page.  You  may  refer  back 
to  previously  completed  evaluations  at  any  time,  but  do  not 
change  any  that  have  already  been  completed.  You  may  remove 
this  page  for  easy  reference  during  the  exercise  if  you  wish. 
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EXAMPLE 


Performance  Evaluation  of  Squadron  Number  X 

S=3atisfactory  U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . _S 


2.  The  quality  of  service  provided  by  this  squadron 

is  rated S 


3.  Compliance  with  AF/AFCS/Area  directives  in  this 

squadron  is  rated  as S 


4.  The  quality  of  maintenance  in  this  squadron  is 

considered  to  be S 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly  [|  1 Highly 

Unsatisfactory  Unsatisfactory  Satisfactory1  Satisfactory 


Performance  Evaluation  of  Squadron  Number  1 


i 


S=Satisf actory  U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  ■ . . S 


2.  The  quality  of  service  provided  by  this  squadron 

is  rated U 


3.  Compliance  with  AF/AFCS/Area  directives  in  this 

squadron  is  rated  as _U 


4.  The  quality  of  maintenance  in  this  squadron  is 

considered  to  be U 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Performance  Evaluation  of  Squadron  Number  2 

S=Satisfactory  U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . _U 


2.  The  quality  of  service  provided  by  this  squadron 

is  rated S 


3.  Compliance  with  AF/AFCS/Area  directives  in  this 

squadron  is  rated  as _S 


4.  The  quality  of  maintenance  in  this  squadron  is 

considered  to  be  ....  S 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Performance  Evaluation  of  Squadron  Number 


S=Satisfactory 


U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . 


2.  The  quality  of  service  provided  by  this  squadron 

is  rated 


3«  Compliance  with  AF/AFCS/Area  directives  in  this 

squadron  is  rated  as U 


4.  The  quality  of  maintenance  in  this  squadron  is 

considered  to  be S 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly 

Highly 

Unsatisfactory  j 

' Unsatisfactory  | 

Satisfactory  | Satisfactory 

Performance  Evaluation  of  Squadron  Number  k_ 


S=Satisfactory 


U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . _U_ 


2.  The  quality  of  service  provided  by  this  squadron 

is  rated S 


3.  Compliance  with  AF/AFCS/Area  directives  in  this 
squadron  is  rated  as  


^ . The  quality  of  maintenance  in  this  squadron  is 

considered  to  be  ............  » U 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly 

Unsatisfactory 


Unsatisfactory 


Satisfactory 


Highly 

Satisfactory  I 


Performance  Evaluation  of  Squadron  Number 


S=Satisfactory  U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . S 


2.  The  quality  of  service  provided  by  this  squadron 

is  rated S 


3.  Compliance  with  AF/AFCS/Area  directives  in  this 

squadron  is  rated  as _3 


4.  The  quality  of  maintenance  in  this  squadron  is 

considered  to  be S 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly 

Unsatisfactory  Unsatisfactory 
L 


Highly 

Satisfactory  j Satisfactory 


Performance  Evaluation  of  Squadron  Number  6 

S=Satisfactory  U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . S 


2.  The  quality  of  service  provided  by  this  squadron 
is  rated  


3.  Compliance  with  AF/AFCS/Area  directives  in  this 
squadron  is  rated  as  


4.  The  quality  of  maintenance  in  this  squadron  is 

considered  to  be U 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Performance  Evaluation  of  Squadron  Number  £ 


S=Satisfactory 


U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . U 


2.  The  quality  of  service  provided  by  this  squadron 


is  rated _U 

3.  Compliance  with  AF/AFCS/Area  directives  in  this 

squadron  is  rated  as __U 


4.  The  quality  of  maintenance  in  this  squadron  is 

considered  to  be  . S 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly 

Unsatisfactory 

Unsatisfactory 

Satisfactory 

Highly 

Satisfactory 

| 

Performance  Evaluation  of  Squadron  Number  8 

S=Satisfactory  U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . S 


2.  The  quality  of  service  provided  by  this  squadron 


is  rated __S 

3.  Compliance  with  AF/AFCS/Area  directives  in  this 

squadron  is  rated  as _S 


The  quality  of  maintenance  in  this  squadron  is 
considered  to  be U 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  ass 


Highly 

Unsatisfactory  Unsatisfactory 


Highly 

Satisfactory  Satisfactory 


Performance  Evaluation  of  Squadron  Number 


S=Satisfactory  U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . _U 


2.  The  quality  of  service  provided  by  this  squadron 


is  rated _S 

3.  Compliance  with  AF/AFCS/Area  directives  in  this 

squadron  is  rated  as _U 


k.  The  quality  of  maintenance  in  this  squadron  is 

considered  to  be S 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly 

Unsatisfactory  Unsatisfactory 


Highly 

Satisfactory  Satisfactory 


Performance  Evaluation  of  Squadron  Number  10 
S=Satisfactory  U=Unsatisfactory 

1.  Personnel  programs  in  this  squadron  are  rated.  . . . S 

2.  The  quality  of  service  provided  by  this  squadron 

is  rated _JL 

3.  Compliance  with  AF/AFCS/Area  directives  in  this 

squadron  is  rated  as S. 

4.  The  quality  of  maintenance  in  this  squadron  is 

considered  to  be U 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly 

Unsatisfactory 

1 

Unsatisfactory  | 

Satisfactory 

Highly 

Satisfactory 

i 

Performance  Evaluation  of  Squadron  Number  11 


S=Satisfactory 


U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . 


2.  The  quality  of  service  provided  by  this  squadron 
is  rated  


Compliance  with  AF/AFCS/Area  directives  in  this 
squadron  is  rated  as 


The  quality  of  maintenance  in  this  squadron  is 
considered  to  be  


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly 

Unsatisfactory  Unsatisfactory 


Highly 

Satisfactory  Satisfactory 


' 

! 

Performance  Evaluation  of  Squadron  Number  12 

S=Satisfactory  U=Unsatisfactory 

1. 

Personnel  programs  in  this  squadron  are  rated.  . . . 

U 

2. 

The  quality  of  service  provided  by  this  squadron 

U 

3- 

Compliance  with  AF/AFCS/Area  directives  in  this 

u 

4. 

The  quality  of  maintenance  in  this  squadron  is 

u 

he 

The  overall  performance  of  this  AFCS  squadron  should 
evaluated  as: 

Highly T | . „ Highly 

Unsatisfactory  [ Unsatisfactory  j Satisfactory  Satisfactory 


Performance  Evaluation  of  Squadron  Number  1 3 


S=Satisfactory 


U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . S 


2.  The  quality  of  service  provided  by  this  squadron 


is  rated 

3.  Compliance  with  AF/AFCS/Area  directives  in  this 

squadron  is  rated  as _S_ 


k.  The  quality  of  maintenance  in  this  squadron  is 

considered  to  be S 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly 

Unsatisfactory 

Unsatisfactory 

Satisfactory 

Highly 

Satisfactory 

Performance  Evaluation  of  Squadron  Number  Ik 

S=Satisfactory  U=Unsatisfactory 

1.  Personnel  programs  in  this  squadron  are  rated.  . . . 

2. 


3. 


k. 


The  quality  of  service  provided  by  this  squadron 
is  rated  


Compliance  with  AF/AFCS/Area  directives  in  this 
squadron  is  rated  as  


The  quality  of  maintenance  in  this  squadron  is 
considered  to  be  


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly 

Unsatisfactory 

Unsati sf actory 

H 

Satisfactory 

Highly 

Satisfactory 

1 

rr  i 


Performance  Evaluation  of  Squadron  Number  15 
S=Satisfactory  U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . S 


2.  The  quality  of  service  provided  by  this  squadron 

is  rated S 


• _JL 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


3.  Compliance  with  AF/AFCS/Area  directives  in  this 
squadron  is  rated  as  


4.  The  quality  of  maintenance  in  this  squadron  is 
considered  to  be  


Highly 

Unsatisfactory  Unsatisfactory 


Highly 

Satisfactory  Satisfactory 


Performance  Evaluation  of  Squadron  Number  16 


S=Satisfactory 


U=Unsatisfactory 


1.  Personnel  programs  in  this  squadron  are  rated.  . . . U 


2.  The  quality  of  service  provided  by  this  squadron 

is  rated S 


Compliance  with  AF/AFCS/Area  dii actives  in  this 
squadron  is  rated  as  ....  


The  quality  of  maintenance  in  this  squadron  is 
considered  to  be _U. 


The  overall  performance  of  this  AFCS  squadron  should 
be  evaluated  as: 


Highly 

Unsatisfactory  Unsatisfactory 


Highly 

Satisfactory  Satisfactory 


DEPARTMENT  OF  THE  AIR  FORCE 
Al«  FORCE  INSTITUTE  OF  TECHNOLOGY  <AU> 
WRIGHT-PATTERSON  AIR  FORCE  BASE.  OHIO  43433 


24  May  1977  ' 

Colonel  Janes  L.  Hodge 
AFCS/LG 

Richards-Gebaur  AF3,  MO  64030 

Dear  Colonel  Hodge: 

Several  weeks  ago  copies  of  the  attached  analysis  exer- 
cise were  distributed  to  over  600  AFCS  middle  managers 
at  the  Headquarters,  three  different  Areas,  and  numer- 
ous units.  The  purpose  of  the  exercise  is  to  capture 
each  individual's  unit  rating  policy,  given  specified 
evaluations  of  sixteen  hypothetical  units  in  terns  of 
four  performance  criteria  common  to  AFCS.  More  than  5 00 
of  the  exercises  have  been  completed  and  returned  to  me. 
The  data  thus  gathered  are  being  analyzed  by  myself  and 
another  AFIT  graduate  student  with  the  intent  of  identi- 
fying, if  possible,  any  differences  in  rater  policies  at 
the  various  levels  of  the  Command. 

While  we  are  concentrating  our  basic  study  on  the  middle 
management  segment  of  AFCS  (0-1  through  0-5  ana  GS-9 
through  GS-13),  we  would  also  like  to  have  a standard 
taken  from  among  the  most  senior  officers  of  the  Command 
with  which  to  compare  the  weights  given  the  performance 
criteria  by  AFCS  middle  managers.  You  can  assist  us  in 
this  effort  if  you  v/ill  take  a few  moments  from  your  busy 
schedule  to  complete  the  attached  exercise,  ignoring  the 
request  for  biographical  data  at  the  beginning.  It  is 
our  intent  to  use  the  data  collected  from  you  and  other 
senior  AFCS  managers  to  calculate  a figure  of  merit  for 
comparison  purposes. 

I realize  that  some  senior  people  might  well  feel  uncom- 
fortable if  asked  to  reveal  their  personal  unit  evaluation 
policies.  In  order  to  assure  that  your  personal  policies 
cannot  be  identified,  I have  asked  the  Head  of  the  AFIT 
Systems  Management  Department,  Colonel  Ronald  A.  Luhks, 
to  act  as  a disinterested  collection  agent  for  the  data 
requested  above.  If  you  will  have  your  secretary  return 
the  completed  attachment  to  him,  Colonel  Luhks  will  re- 
move any  identifying  information  and  turn  only  the  steri- 
lized results  over  to  me. 

This  study  has  been  conducted  under  AFCS  sponsorship  and 
the  results  will  be  presented  to  AFCS/OA  in  the  form  of  a 
graduate  thesis  which  I hope  to  complete  by  early  July. 

I honestly  believe  that  we  can  help  to  improve  understand- 
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ing  of  the  complex  interfaces  between  the  different  AFCS 
command  levels  through  this  research  effort.  Please  try 
to  find  the  time  to  provide  us  with  your  valuable  input 
to  this  effort  and  have  your  secretary  return  the  com- 
pleted attachment  to: 

Colonel  R.A.  Luhks 

AFIT/ENS 

Wright-Patterson  AF3,  OH  ^5^33 
Sincerely, 


CARL  G.  0 ' 3ERRY,  Lt  Col,  USAF 
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Please  indicate  in  the  space  provided  next  to  each 
performance  criterion  the  importance  you  feel  should  be 
associated  with  that  factor  when  assessing  the  overall 
performance  of  an  AFCS  unit.  Use  a total  of  100  points 
as  indicated  (the  more  important  you  feel  a factor  is, 
the  more  points  it  should  receive) : 


Personnel  Programs  (the  success  of  the  unit  in 
terms  of  morale  and  welfare  programs,  disciplin- 
ary rates,  IG  complaint  rate,  retention  rate). 


Quality  of  Service  (unit  success  in  factors 
related  to  the  service  provided  to  the  cus- 
tomer) . 


Compliance  (the  degree  to  which  the  unit 
adheres  to  higher  headquarters  directives) . 


Maintenance  Quality  (maintenance  effective' 
ness  and  efficiency) . 


Total  100 


Now  please  proceed  with  the  remainder  of  the  ex- 
ercise. 
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APPENDIX  C 


FEEDBACK  PROVIDED  TO  SUBJECTS 

Each  experimental  subject  was  afforded  the  opportunity 
to  request  an  analysis  of  his  individual  performance  in  the 
decision  exercise,  as  an  incentive  to  improve  the  response 
rate.  A computer  program  was  prepared  to  generate  the  in- 
dicated feedback  to  the  subject,  as  shown  in  this  Appendix-. 
The  computer  program  selected  the  subject  on  the  basis  of  a 
coded  data  entry,  calculated  his  individual  relative  pre- 
dictor  weights  and  consistency  index  (R  ),  and  printed  the 
indicated  message.  The  analyses  were  mailed  in  bulk  to  the 
appropriate  units  for  distribution  to  participants  who  had 
requested  the  feedback. 
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T4PT  J.P.  MILLFR 
AFIT/rnS 

WR  ir.Mr-PATTFPSON  AFP,  OH 


APPENDIX  D 

EXAMPLE  OF  SPSS  PROGRAM  AND  TYPICAL  OUTPUT 

This  Appendix  contains  an  example  of  the  control 
cards  required  as  input  for  the  Statistical  Package  for  the 
Social  Sciences  (SPSS)  regression  algorithm  as  used  in  this 
study.  Also  included  is  a typical  last  regression  step 
output,  showing  the  Analysis  of  Variance  (ANOVA)  table  and 
statistics  produced  by  the  algorithm. 

Since  513  AFCS  middle  managers  responded  to  this  ex- 
periment, there  were  513  x 16  = 8,208  separate  evaluations 
made.  Each  of  these  decisions  is  included  in  the  overall 
regression  model,  then. the  respondents  are  broken  out  by 
headquarters  and  Area-grouped  operating  units  for  separate 
regression  runs.  The  average  regression  after  this  grouping 
process  included  more  than  2000  evaluations;  hence  each 
model  thus  produced  was  quite  insensitive  to  the  effects  of 
extremes  in  individual  evaluation  cases.  The  data  base  was 
screened  for  obvious  attempts  to  introduce  inaccuracies 
(marking  all  entries  high  or  low,  choosing  all  the  same 
values,  etc.),  but  none  were  detected.  If  any  subject  elec- 
ted to  simply  mark  his  exercise  randomly  this  would  not 
materially  affect  the  regression  outcome,  with  such  large 
numbers  of  decisions  being  included  in  each  regression  run. 
It  was  apparent  that  some  respondents  used  models  to  make 

their  evaluations,  since  there  were  six  cases  out  of  513 

2 

where  ther  individual  R was  1.000. 
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APPENDIX  E 


CALCULATION  OF  F-TEST  VALUES 


The  F-Test  values  used  to  compare  regression  models 
in  this  study  were  calculated  using  the  following  formula: 


F0  = 


[sse  - jiiSSgjll/CCp-iHk+i)] 

C jljSSe j]/Cn-p(k+l ) ] 


where  SSe  is  the  residual  sum  of  squares  derived  by  regres- 
sing all  compared  groups  of  decisions  together,  SSgj  is  the 
residual  sum  of  squares  for  the  jth  group  of  evaluations,  p 
is  the  number  of  groups  being  compared  (number  of  subsets  of 
data  in  the  regression),  k is  the  number  of  predictor  vari- 
ables (four,  in  all  cases  for  this  study),  and  n is  the  total 
number  of  decisions  in  all  groups  being  compared. 

The  null  hypothesis  being  tested  is: 

Q 0 

P 1 

H0:  3 j = §_2  = £3  = = £p»  where  3^  = 


The  alternate  hypothesis  is: 

H '•  if  t £ j > for  at  least  one  i,j  pair. 

The  null  hypothesis  is  rejected  if: 

Fo  > Fot,  [ (p-1 ) (k+1 ) ] , [n-p(k+l ) ] 
where  a = 0.05  in  all  comparisons  made  for  this  study. 
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